MurreNet通过解耦病理与基因数据的交互关系,提升癌症生存预测准确率。
MurreNet: Modeling Holistic Multimodal Interactions Between Histopathology and Genomic Profiles for Survival Prediction
- 分离病理与基因数据的特有和共享特征,减少冗余
- 在六个人类癌种数据集上达到当前最优生存预测效果
- 适合关注多模态融合与精准医疗的研究者
癌症生存预测需整合病理全切片图像(WSIs)与基因组数据,但因数据异质性及模态间/内交互复杂,现有方法常采用简单融合策略,未能充分捕捉模态特异与共通交互,导致对多模态关联理解不足、预测性能受限。本文提出多模态表示解耦网络(MurreNet),首先设计多模态表示分解(MRD)模块,将配对输入显式分解为模态特异与模态共享表示,降低模态间冗余;进一步通过新型训练正则化策略约束特征分布的相似性、差异性与代表性,精炼并更新解耦表示;最后利用深度整体正交融合(DHOF)策略整合增强后的多模态特征,构建联合表示。在六个TCGA癌种队列上的大量实验表明,MurreNet在生存预测任务中达到当前最优(SOTA)性能。
原文摘要 · Abstract (English)
Cancer survival prediction requires integrating pathological Whole Slide Images (WSIs) and genomic profiles, a challenging task due to the inherent heterogeneity and the complexity of modeling both inter- and intra-modality interactions. Current methods often employ straightforward fusion strategies for multimodal feature integration, failing to comprehensively capture modality-specific and modality-common interactions, resulting in a limited understanding of multimodal correlations and suboptimal predictive performance. To mitigate these limitations, this paper presents a Multimodal Representation Decoupling Network (MurreNet) to advance cancer survival analysis. Specifically, we first propose a Multimodal Representation Decomposition (MRD) module to explicitly decompose paired input data into modality-specific and modality-shared representations, thereby reducing redundancy between modalities. Furthermore, the disentangled representations are further refined then updated through a novel training regularization strategy that imposes constraints on distributional similarity, difference, and representativeness of modality features. Finally, the augmented multimodal features are integrated into a joint representation via proposed Deep Holistic Orthogonal Fusion (DHOF) strategy. Extensive experiments conducted on six TCGA cancer cohorts demonstrate that our MurreNet achieves state-of-the-art (SOTA) performance in survival prediction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。