融合病理与基因数据,用Mamba高效分析癌症生存期。
ME-Mamba: Multi-Expert Mamba with Efficient Knowledge Capture and Fusion for Multimodal Survival Analysis
- 分设病理与基因专家,用Mamba捕捉长序列特征
- 通过最优传输和最大均值差异实现跨模态融合
- 在TCGA五数据集上达到领先性能,计算开销低
基于全幻灯片图像(WSIs)的生存分析在癌症研究中至关重要。尽管取得显著进展,病理图像通常仅提供切片级标签,难以从千兆像素级WSI中学习判别性表征。随着高通量测序技术的快速发展,整合病理图像与基因组数据的多模态生存分析成为新趋势。本文提出多专家Mamba(ME-Mamba)系统,能有效捕捉病理与基因组特征,并实现两者的高效融合。该方法在不丢失单模态关键信息的前提下,实现互补信息融合,从而提升癌症生存分析精度。具体而言,首先引入病理专家与基因组专家分别处理单模态数据,二者均采用结合常规扫描与注意力扫描机制的Mamba架构,以提取长序列中具有判别性的特征。其次,设计协同专家负责模态融合,通过最优传输显式学习跨模态的令牌级局部对应关系,并利用基于最大均值差异的全局交叉模态融合损失隐式增强分布一致性。融合后的特征表示随后输入Mamba主干网络进行进一步整合。通过病理专家、基因组专家与协同专家的协作,本方法在保持相对较低计算复杂度的同时,实现了稳定且精确的生存分析。在癌症基因组图谱(TCGA)五个数据集上的大量实验表明,该方法达到当前最优性能。
原文摘要 · Abstract (English)
Survival analysis using whole-slide images (WSIs) is crucial in cancer research. Despite significant successes, pathology images typically only provide slide-level labels, which hinders the learning of discriminative representations from gigapixel WSIs. With the rapid advancement of high-throughput sequencing technologies, multimodal survival analysis integrating pathology images and genomics data has emerged as a promising approach. We propose a Multi-Expert Mamba (ME-Mamba) system that captures discriminative pathological and genomic features while enabling efficient integration of both modalities. This approach achieves complementary information fusion without losing critical information from individual modalities, thereby facilitating accurate cancer survival analysis. Specifically, we first introduce a Pathology Expert and a Genomics Expert to process unimodal data separately. Both experts are designed with Mamba architectures that incorporate conventional scanning and attention-based scanning mechanisms, allowing them to extract discriminative features from long instance sequences containing substantial redundant or irrelevant information. Second, we design a Synergistic Expert responsible for modality fusion. It explicitly learns token-level local correspondences between the two modalities via Optimal Transport, and implicitly enhances distribution consistency through a global cross-modal fusion loss based on Maximum Mean Discrepancy. The fused feature representations are then passed to a mamba backbone for further integration. Through the collaboration of the Pathology Expert, Genomics Expert, and Synergistic Expert, our method achieves stable and accurate survival analysis with relatively low computational complexity. Extensive experimental results on five datasets in The Cancer Genome Atlas (TCGA) demonstrate our state-of-the-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。