解决阿尔茨海默病诊断中多模态数据缺失问题,提升模型鲁棒性。
PRA-PoE: Robust Multimodal Alzheimer's Diagnosis with Arbitrary Missing Modalities

- 用原型对齐与不确定性感知融合机制,显式建模模态缺失与置信度。
- 在真实缺失场景下测试,平均准确率提升5.4%,F1值提升10.9%。
- 适合临床多模态诊断系统,尤其应对数据不完整的情况。
现实中的阿尔茨海默病评估常面临多模态数据缺失,且训练与部署时的缺失模式不一致,导致条件表示偏移。现有方法依赖隐式填补或模态合成,难以显式建模模态可用性与不确定性,造成过度自信、鲁棒性差及不确定性估计失准。为此,本文提出PRA-PoE框架,包含原型锚定表征对齐(PRA)和不确定性感知专家乘积(UA-PoE)融合机制。PRA通过可学习全局原型与可用性条件令牌,区分已观测与缺失模态,重构缺失特征,并自适应调整观测表示以对齐不同模态子集的潜在空间,缓解表示偏移。UA-PoE将各模态建模为高斯专家,采用闭式专家乘积融合,不确定性高的专家因精度低自动降权,提升不确定性可靠性。在临床真实协议下,使用自然缺失数据训练并在所有非空模态组合上测试,PRA-PoE在ADNI数据集上平均准确率相对最优基线提升5.4%,在OASIS-3上平均F1值提升10.9%,在所有非空模态子集上均表现最优。
原文摘要 · Abstract (English)
Missing modalities are prevalent in real-world Alzheimer's disease (AD) assessment and pose a significant challenge to multimodal learning, particularly when the distribution of observed modality subsets differs between training and deployment. Such missingness pattern mismatch induces a conditional representation shift across modality subsets. Existing approaches that rely on implicit imputation or modality synthesis often fail to explicitly model modality availability and uncertainty, leading to overconfident dependence on synthesized features, reduced robustness, and miscalibrated uncertainty estimates. To address these limitations, we propose PRA-PoE, an incomplete multimodal learning framework that is equipped with Prototype-anchored Representation Alignment (PRA) and an Uncertainty-aware Product of Experts (UA-PoE) fusion mechanism. First, PRA uses learnable global prototypes and availability-conditioned tokens to encode modality availability, distinguish observed from missing modalities, re-synthesize features for missing modalities, and adaptively refine observed representations to align latent spaces across modality subsets, with the goal of reducing representation shift under varying missingness patterns. Second, UA-PoE models each modality as a Gaussian expert and performs closed-form Product of Experts fusion, where experts with higher uncertainty are automatically down-weighted via lower precision, improving uncertainty reliability. We evaluate PRA-PoE under a clinically realistic protocol by training with naturally missing data and testing on all non-empty modality combinations. PRA-PoE consistently outperforms the state-of-the-art across datasets, achieving a 5.4% relative improvement in average accuracy on ADNI and a 10.9% relative gain in average F1 on OASIS-3 over the strongest baseline across all non-empty modality subsets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。