arXiv:2511.18089cs.CV2025-11

提出新框架平衡多模态数据共性与特异性,提升癌症生存预测准确率。

Together, Then Apart: Balancing Alignment and Distinctiveness for Multimodal Survival Analysis

  • 先对齐共享特征,再强化模态特异性,实现‘先合后分’的表示学习
  • 在TCGA多个癌种数据集上显著优于现有模型,提升生存预测性能
  • 可解释的原型结构揭示了跨模态临床关联,适合医学影像与基因组联合分析

多模态生存分析旨在利用组织病理图像和基因组谱等异构生物医学数据改善癌症预后。通常策略是跨模态对齐以捕捉共享信号,但过度对齐会削弱对生存预测至关重要的模态特异性信息。本文基于一个简单观察:有效模型应先发现模态间共享模式,再保留各自特异性。据此提出TTA框架,通过原型对齐捕获跨模态生存相关结构,并采用锚点引导对比损失增强模态特异性。为应对模态不平衡与噪声对应问题,引入非平衡最优传输建模跨模态交互。在包含配对病理与基因组数据的多个TCGA癌种队列上评估,TTA始终优于近期多模态生存模型,且学习到的原型结构揭示了与临床结局相关的可解释跨模态模式。

原文摘要 · Abstract (English)

Multimodal survival analysis aims to improve cancer prognosis using heterogeneous biomedical data, such as histopathology images and genomic profiles. A common strategy is to align representations across modalities so that shared signals can be captured. However, strong cross-modal alignment can also remove modality-specific evidence that is critical for survival prediction. In this paper, we revisit multimodal survival learning from a simple observation: effective models should first discover shared patterns across modalities, and then preserve modality-specific signals. This motivates a representation learning principle that we refer to as Together Then Apart. Based on this idea, we propose TTA, a framework that balances cross-modal alignment and representation distinctiveness. TTA first performs prototype-based alignment to capture shared survival-related structures between modalities. It then encourages modality-specific distinctiveness through an anchor-guided contrastive objective. To further account for modality imbalance and noisy correspondences, we model cross-modal interactions using unbalanced optimal transport. We evaluate the proposed approach on multiple TCGA cancer cohorts with paired histopathology and genomic data. TTA consistently improves survival prediction over recent multimodal survival models. Moreover, the learned prototype structures reveal interpretable cross-modal patterns associated with clinical outcomes.

多模态学习生存分析可解释性癌症预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。