提出轻量级特征补全网络,解决医疗多模态联邦学习中缺失模态问题
Multimodal Federated Learning With Missing Modalities through Feature Imputation Network
- 设计低维特征翻译器,重建缺失模态的瓶颈特征
- 在三个数据集上均提升基线模型性能,跨异构场景稳定有效
- 无需生成合成数据,适合资源受限的医疗联邦学习场景
多模态联邦学习在不共享原始数据的前提下协同训练模型,可缓解医疗领域数据稀缺与隐私问题。但临床实践中因诊疗差异、成本限制、数据回溯收集、隐私顾虑或技术/人为错误,常出现模态缺失。现有方法依赖公开真实数据或合成数据补全,但获取特定疾病的真实数据不现实,且高维医学数据的生成模型训练成本高、易出错。本文提出一种新型轻量级、低维特征翻译器,用于重建缺失模态的瓶颈特征。在MIMIC-CXR、NIH Open-I和CheXpert三个数据集上,无论同质还是异构设置,均持续提升多个竞争性基线的性能。代码与实现细节见:https://github.com/bhattarailab/FedFeatGen
原文摘要 · Abstract (English)
Multimodal federated learning holds immense potential for collaboratively training models from multiple sources without sharing raw data, addressing both data scarcity and privacy concerns, two key challenges in healthcare. A major challenge in training multimodal federated models in healthcare is the presence of missing modalities due to multiple reasons, including variations in clinical practice, cost and accessibility constraints, retrospective data collection, privacy concerns, and occasional technical or human errors. Previous methods typically rely on publicly available real datasets or synthetic data to compensate for missing modalities. However, obtaining real datasets for every disease is impractical, and training generative models to synthesize missing modalities is computationally expensive and prone to errors due to the high dimensionality of medical data. In this paper, we propose a novel, lightweight, low-dimensional feature translator to reconstruct bottleneck features of the missing modalities. Our experiments on three different datasets (MIMIC-CXR, NIH Open-I, and CheXpert), in both homogeneous and heterogeneous settings consistently improve the performance of competitive baselines. The code and implementation details are available at: https://github.com/bhattarailab/FedFeatGen
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。