解决多模态联邦学习中数据缺失问题，通过原型引导专家路由合成缺失特征。
ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
- 用原型库捕捉跨机构的模态先验，指导专家网络动态路由。
- 在4个胸片数据集上，无论同质还是异质设置均优于现有方法。
- 适合医疗多模态联邦学习场景，尤其适用于模态缺失严重的实际应用。
本文针对多模态联邦学习中的模态缺失问题，提出ProMoE-FL——一种原型条件化的专家混合框架，用于鲁棒地合成缺失模态特征。现有方法依赖外部公共数据集或仅基于可用模态进行简单特征合成，存在局限性。ProMoE-FL构建全局客户端感知的原型库，捕捉跨机构的临床有意义模态先验；其专家混合模型根据原型和模态索引进行条件化路由，实现方向感知的动态特征合成。我们在四个公开胸片数据集（MIMIC-CXR、NIH Open-I、PadChest、CheXpert）上进行了广泛的定量与定性评估，结果表明，无论在同质还是更具挑战性的异质设置下，ProMoE-FL均持续优于当前最优方法。
原文摘要 · Abstract (English)
In this paper, we address the problem of multimodal federated learning with missing modality. Existing methods utilize an additional public dataset or perform naive feature synthesis that is based solely on the available modality. To address these limitations, we propose ProMoE-FL, a Prototype-conditioned Mixture-of-Experts framework for robust missing-modality feature synthesis in multimodal federated learning. ProMoE-FL builds a global client-aware prototype bank that captures clinically meaningful modality priors across institutions. Our Mixture of Experts is conditioned on these prototypes and modality indices to enable direction-aware expert routing for dynamically synthesizing missing features. We perform extensive quantitative and qualitative evaluations on four public chest X-ray datasets (MIMIC-CXR, NIH Open-I, PadChest, and CheXpert) and demonstrate that ProMoE-FL consistently outperforms state-of-the-art methods in both homogeneous as well as the more challenging heterogeneous settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。