arXiv:2607.09982cs.LG2026-07中稿 · CHASE 2026

提出可解释的多模态路由框架,提升临床预测透明度与鲁棒性。

Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction

论文配图:Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction
图 1 · 摘自论文原文
  • 构建单模、双模、三模路径,显式建模各数据源贡献
  • 在MIMIC-IV上实现25个表型和重症死亡预测,性能稳定
  • 支持推理时掩码测试,无需重训即可审计模型决策

电子健康记录(EHR)数据具有多模态特性,融合多种模态可提升预测性能。然而,现有方法多依赖深度融合,难以揭示各模态对预测的具体贡献,限制了可解释性。本文提出一种显式的多模态路由框架,用于临床预测,涵盖结构化纵向变量(L)、临床文本(N)和胸部X光片(I)三种模态。模型构建离散的单模、定向双模及三模路径,捕捉各模态信号及非对称跨模态交互。为审计多模态推理并评估鲁棒性,引入推理时路径掩码机制,模拟模态缺失并重加权剩余路径,无需重训练。通过分析性能变化与路由权重,揭示模型决策逻辑。在MIMIC-IV数据集上,针对25个表型的多标签预测和二分类重症监护室死亡预测任务进行评估,发现不同疾病组别中模态依赖存在系统性差异。整体框架提供透明、可审计、实用的多模态临床预测方案,兼具可解释性、鲁棒性及对数据源作用的洞察。

原文摘要 · Abstract (English)

Electronic health record (EHR) data are inherently multimodal, and leveraging multiple modalities can improve predictive performance. However, most existing approaches rely on deep fusion, which obscures how individual modalities contribute to predictions and limits the interpretability of multimodal reasoning. We propose an explicit multimodal routing framework for clinical prediction that enables interpretable, robust, and auditable reasoning across three EHR modalities: structured longitudinal variables (L), clinical notes (N), and chest X-rays (I). Our model constructs discrete unimodal, directional bimodal, and trimodal routes to capture both individual modality signals and asymmetric cross-modal interactions. To audit multimodal reasoning and assess robustness, we introduce inference-time route masking, which simulates missing modalities and reweights the remaining routes without retraining. We analyze changes in performance and routing weights under these scenarios to understand model decision-making. We evaluate our framework on multi-label phenotype prediction (K = 25) and binary ICU mortality prediction using trimodal patient stays from MIMIC-IV, revealing systematic differences in modality reliance across clinical condition groups. Overall, our framework offers a transparent, auditable, and practical approach to multimodal clinical prediction, providing interpretability, robustness, and insights into how different data sources drive model decisions.

多模态学习临床预测可解释性医疗AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。