融合病理与基因数据,自适应预测生存期,缺损数据也能准
AdaMHF: Adaptive Multimodal Hierarchical Fusion for Survival Prediction
- 分层自适应融合多模态特征,捕捉跨模态细粒度交互
- 在TCGA数据集上,完整与缺失模态下均超越现有最佳方法
- 专为医学数据设计,资源消耗低,适合真实临床场景
病理图像与基因组数据的联合分析在生存预测中日益受到关注。然而,现有方法常忽略生物特性中的异质性与稀疏性,限制了其临床适用性。为此,我们提出AdaMHF:自适应多模态分层融合框架,实现高效、全面且定制化的特征提取与融合。该框架针对医学数据特点,即使在缺失模态等挑战条件下,仍能以低资源消耗实现精准预测。首先,通过专家扩展与残差结构激活专用专家,提取异质性和稀疏特征;随后对提取的特征进行选择与聚合,降低非主导特征权重,同时保留完整信息。接着,采用分层融合机制,捕获跨模态的多粒度交互。此外,我们构建了一个模拟真实临床条件的生存预测基准,专门应对模态缺失问题。在TCGA数据集上的大量实验表明,AdaMHF在完整与不完整模态设置下均优于当前最优方法。
原文摘要 · Abstract (English)
The integration of pathologic images and genomic data for survival analysis has gained increasing attention with advances in multimodal learning. However, current methods often ignore biological characteristics, such as heterogeneity and sparsity, both within and across modalities, ultimately limiting their adaptability to clinical practice. To address these challenges, we propose AdaMHF: Adaptive Multimodal Hierarchical Fusion, a framework designed for efficient, comprehensive, and tailored feature extraction and fusion. AdaMHF is specifically adapted to the uniqueness of medical data, enabling accurate predictions with minimal resource consumption, even under challenging scenarios with missing modalities. Initially, AdaMHF employs an experts expansion and residual structure to activate specialized experts for extracting heterogeneous and sparse features. Extracted tokens undergo refinement via selection and aggregation, reducing the weight of non-dominant features while preserving comprehensive information. Subsequently, the encoded features are hierarchically fused, allowing multi-grained interactions across modalities to be captured. Furthermore, we introduce a survival prediction benchmark designed to resolve scenarios with missing modalities, mirroring real-world clinical conditions. Extensive experiments on TCGA datasets demonstrate that AdaMHF surpasses current state-of-the-art (SOTA) methods, showcasing exceptional performance in both complete and incomplete modality settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。