arXiv:2606.06867cs.CV2026-06

解决癌症诊断中多模态数据缺失问题,提升模型鲁棒性与准确性。

Multi-FRuGaL: Multimodal Flexible Redundancy-aware Decomposed Gated Learning for Cancer Diagnosis and Prognosis

论文配图:Multi-FRuGaL: Multimodal Flexible Redundancy-aware Decomposed Gated Learning for Cancer Diagnosis and Prognosis
图 1 · 摘自论文原文
  • 通过分解与门控机制分离模态特异信号与冗余信息。
  • 在数据缺失情况下仍保持高精度,生存预测AUC达0.8496。
  • 适合临床真实场景下多源医疗数据融合,尤其适用于数据不全的病例。

现代医学依赖于影像、病理、文本报告和结构化临床信息等异构数据源。然而真实患者数据常存在缺失或稀疏采集的模态,限制了传统多模态融合方法的效果。为此,我们提出多模态灵活冗余感知分解门控学习框架(Multi-FRuGaL),一种基于分解感知的自适应门控中间融合方法,可在模态缺失条件下进行模态级表征学习。该框架结合各模态编码器、信号分解层、输入条件门控网络及信息感知融合目标,有效分离冗余与互补信号,选择性增强有用模态,抑制冗余或噪声输入,即使多个模态缺失也能保持稳定性能。我们在两个头颈部癌队列上评估:HANCOCK挑战数据集(N=763,包含五种模态,两个预后终点:5年生存率与2年复发率)和HECKTOR挑战数据集(N=588,三种模态用于人乳头瘤病毒(HPV)状态分类)。Multi-FRuGaL在多项任务中均优于基线,生存预测AUC从0.601提升至0.8496,复发预测从0.672升至0.8102,HPV分类达0.975 AUC。在生存分析中,其整体生存、无复发生存和无进展生存的C-index分别为0.6814、0.7421和0.7143(HANCOCK),HECKTOR上无复发生存C-index为0.7203。定性分析显示,即使在严重模态缺失条件下,该模型仍能学习到判别性强且稳健的多模态表示。

原文摘要 · Abstract (English)

Modern medicine relies on heterogeneous data sources spanning radiology, pathology, text reports, and structured clinical information. However, real-world patient data are frequently incomplete, with missing or sparsely acquired modalities, limiting the effectiveness of standard multimodal fusion approaches. To this end, we propose the Multimodal Flexible Redundancy-aware decomposed GAted Learning (Multi-FRuGaL) framework, a decomposition-aware, adaptive gated intermediate-fusion framework that performs modality-level representation learning under missing data. Multi-FRuGaL integrates per-modality encoders with a signal decomposition layer, an input-conditioned gating network, and an information-aware fusion objective to separate redundant from modality-specific complementary signals, selectively upweighting informative modalities and suppressing redundant or noisy inputs, and remaining well-defined even when multiple modalities are absent. We evaluate Multi-FRuGaL on two multimodal head and neck cancer cohorts: the HANCOCK challenge dataset (N = 763) comprising five modalities and two prognostic endpoints (5-year survival and 2-year recurrence), and the HECKTOR challenge dataset (N = 588) comprising three modalities for human papillomavirus (HPV) status classification. Multi-FRuGaL consistently achieves higher mean performance than the evaluated baselines across multiple tasks, improving AUC from 0.601 to 0.8496 for survival, from 0.672 to 0.8102 for recurrence, and achieving 0.975 AUC for HPV prediction on HECKTOR. For survival analysis, it further achieves a concordance index of 0.6814 for overall survival, 0.7421 for recurrence-free survival, and 0.7143 for progression-free survival on HANCOCK, and 0.7203 for recurrence-free survival on HECKTOR. Qualitative analyses further show that Multi-FRuGaL learns discriminative and robust multimodal representations, even under severe missing-modality conditions.

多模态学习癌症诊断数据缺失医疗AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。