基于可信度的专家路由,提升多模态医学分类的可靠性与泛化能力。
TIER-MoE: Trust-Informed Expert Routing via Conditional Modality Risk for Multimodal Fusion in Biomedical Classification

- 用交叉验证预测损失衡量模态可信度,动态分配专家路由。
- 在4个生物医学数据集上均超越当前最佳,零样本外推表现优异。
- 适合需要高可信推理的医学诊断场景,尤其关注模态可靠性评估。
多模态融合的潜力在于整合互补证据,但更多证据未必带来更好预测。现有模型虽通过增强跨模态交互和样本自适应融合取得进展,但模态在融合中的权重无法反映其是否不可靠、冗余或与特定专家不匹配。为此,我们提出TIER-MoE,一种基于风险引导的子空间混合专家模型,将样本级模态可靠性定义为该模态单模预测器预期产生的预测损失,该风险由未使用对应样本训练的模型生成的留置外预测学习而来。TIER-MoE将估计的风险与专家子空间兼容性结合,实现稀疏的模态-专家路由,同时保留始终激活的共享路径以维持多模态互补性。我们在四个公开的多模态生物医学数据集上评估了TIER-MoE,涵盖阿尔茨海默病状态、皮肤病变恶性程度及视网膜分类。结果表明,其在预测性能和概率校准方面优于现有最先进方法,在宏平均F1和Brier得分上均有持续提升,并展现出对外部队列的强大零样本泛化能力。
原文摘要 · Abstract (English)
The promise of multimodal fusion lies in combining complementary sources of evidence, yet more evidence does not always yield a better prediction. Recent multimodal models have advanced fusion through richer cross-modal interaction and sample-adaptive fusion. However, the influence assigned to a modality during fusion does not reveal whether that source is unreliable, redundant, or poorly matched to a specialized expert. To address this limitation, we introduce TIER-MoE, a risk-guided subspace mixture-of-experts model that defines sample-specific modality reliability as the prediction loss its unimodal predictor is expected to incur. This risk is learned from out-of-fold predictions generated by models that were not trained on the corresponding sample. TIER-MoE combines the estimated risk with expert-specific subspace compatibility for sparse modality-expert routing, while an always-active shared path preserves multimodal complementarity. We evaluate TIER-MoE on four public multimodal biomedical datasets spanning Alzheimer's disease status, skin-lesion malignancy, and retinal classification. Results demonstrate its superiority over state-of-the-art methods in predictive performance and probability calibration, with consistent improvements in Macro-F1 and Brier score and strong zero-shot generalization to an external cohort.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。