针对医学多模态数据缺失问题,提出可分离特异性信息的新型模型。
MUST: Modality-Specific Representation-Aware Transformer for Diffusion-Enhanced Survival Prediction with Missing Modality
- 通过低秩共享空间分解模态特有与共现信息
- 在病理或基因数据缺失时仍保持高精度生存预测
- 适合临床实际中数据不全的癌症预后场景
从多模态医学数据中准确预测生存期对精准肿瘤学至关重要,但临床应用常受限于模态缺失——因成本、技术或回顾性数据获取限制。现有方法尝试通过特征对齐或联合分布学习缓解此问题,却未显式建模各模态的独特贡献。我们提出MUST(Modality-Specific representation-aware Transformer),在学习到的低秩共享子空间中,通过代数约束显式将每种模态表示分解为模态特有与跨模态上下文成分。该分解可精确识别某模态缺失时丢失的信息。对于无法从其他模态推断出的真正模态特有信息,采用条件潜变量扩散模型,基于恢复的共享信息与学习到的结构先验生成高质量表示。在五个TCGA癌症数据集上的实验表明,MUST在完整数据下达到当前最优性能,并在病理或基因数据缺失条件下仍保持稳健预测,推理延迟处于临床可接受范围。
原文摘要 · Abstract (English)
Accurate survival prediction from multimodal medical data is essential for precision oncology, yet clinical deployment faces a persistent challenge: modalities are frequently incomplete due to cost constraints, technical limitations, or retrospective data availability. While recent methods attempt to address missing modalities through feature alignment or joint distribution learning, they fundamentally lack explicit modeling of the unique contributions of each modality as opposed to the information derivable from other modalities. We propose MUST (Modality-Specific representation-aware Transformer), a novel framework that explicitly decomposes each modality's representation into modality-specific and cross-modal contextualized components through algebraic constraints in a learned low-rank shared subspace. This decomposition enables precise identification of what information is lost when a modality is absent. For the truly modality-specific information that cannot be inferred from available modalities, we employ conditional latent diffusion models to generate high-quality representations conditioned on recovered shared information and learned structural priors. Extensive experiments on five TCGA cancer datasets demonstrate that MUST achieves state-of-the-art performance with complete data while maintaining robust predictions in both missing pathology and missing genomics conditions, with clinically acceptable inference latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。