解决肺癌生存预测中多模态数据缺失问题,提升模型鲁棒性。
Handling Missing Modalities in Multimodal Survival Prediction for Non-Small Cell Lung Cancer
- 采用基础模型提取多模态特征,支持不完整数据下的中间融合。
- 三模态联合预测达到C-index 74.42,优于单模态及早期/晚期融合。
- 自动调整各模态权重,适合临床实际中数据不全的场景。
非小细胞肺癌(NSCLC)的精准生存预测需整合临床、影像与病理数据。多模态深度学习(MDL)可提升预后精度,但小样本和模态缺失限制其临床应用,传统方法依赖完整病例筛选或数据填补。本文提出一种对缺失敏感的多模态生存预测框架,结合计算机断层扫描(CT)、全切片病理图像(WSI)与结构化临床变量,针对不可切除Ⅱ-Ⅲ期NSCLC进行总生存建模。该框架利用基础模型(FMs)进行模态特异性特征提取,并采用对缺失敏感的编码策略,在自然不完整的模态配置下实现中间层次的多模态融合。模型设计确保训练与推理过程中无需剔除患者。中间融合优于单模态基线及早/晚融合策略,三模态配置达到C-index 74.42。模态重要性分析显示,融合模型根据特征表达信息量动态调整各数据流依赖度,受基础模型预训练目标与生存任务对齐程度影响。学习到的风险评分能实现疾病进展与转移风险的临床有意义分层,所有模态组合均通过统计显著的log-rank检验,验证了该框架的转化价值。
原文摘要 · Abstract (English)
Accurate survival prediction in Non-Small Cell Lung Cancer (NSCLC) requires integrating clinical, radiological, and histopathological data. Multimodal Deep Learning (MDL) can improve precision prognosis, but small cohorts and missing modalities limit its clinical applicability, as conventional approaches enforce complete case filtering or imputation. We present a missing-aware multimodal survival framework that combines Computed Tomography (CT), Whole-Slide Histopathology Images (WSI), and structured clinical variables for overall survival modeling in unresectable stage II-III NSCLC. The framework uses Foundation Models (FMs) for modality-specific feature extraction and a missing-aware encoding strategy that enables intermediate multimodal fusion under naturally incomplete modality profiles. By design, the architecture processes all available data without dropping patients during training or inference. Intermediate fusion outperforms unimodal baselines and both early and late fusion strategies, with the trimodal configuration reaching a C-index of 74.42. Modality-importance analyses show that the fusion model adapts its reliance on each data stream according to representation informativeness, shaped by the alignment between FM pretraining objectives and the survival task. The learned risk scores produce clinically meaningful stratification of disease progression and metastatic risk, with statistically significant log-rank tests across all modality combinations, supporting the translational relevance of the proposed framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。