解决联邦学习中多模态数据部分缺失问题，提升模型泛化能力。
FedMAC: Tackling Partial-Modality Missing in Federated Learning with Cross-Modal Aggregation and Contrastive Regularization
- 设计跨模态聚合与对比正则化机制，应对实例级数据缺失差异。
- 在严重缺失场景下性能超越基线26%，显著提升模型鲁棒性。
- 适用于医疗、智能设备等多模态数据分布不均的联邦场景。
联邦学习（FL）通过分布式数据协作训练共享模型，保护本地数据隐私。然而，在客户端数据存在模态缺失时，特别是部分模态缺失导致的实例级异质性问题，现有方法难以应对——缺失模式在不同样本间差异显著。为此，本文提出新型框架FedMAC，专门处理联邦学习中的多模态部分缺失问题。为防止多模态特征被平凡聚合，引入基于对比学习的正则化，约束潜在表示空间。实验表明，在多种客户端配置和统计异质性条件下，FedMAC表现优异，严重缺失场景下性能较基线最高提升26%，验证了其在解决部分模态缺失问题上的有效性。代码已开源：https://github.com/nmduonggg/PEPSY。
原文摘要 · Abstract (English)
Federated Learning (FL) is a method for training machine learning models using distributed data sources. It ensures privacy by allowing clients to collaboratively learn a shared global model while storing their data locally. However, a significant challenge arises when dealing with missing modalities in clients' datasets, where certain features or modalities are unavailable or incomplete, leading to heterogeneous data distribution. While previous studies have addressed the issue of complete-modality missing, they fail to tackle partial-modality missing on account of severe heterogeneity among clients at an instance level, where the pattern of missing data can vary significantly from one sample to another. To tackle this challenge, this study proposes a novel framework named FedMAC, designed to address multi-modality missing under conditions of partial-modality missing in FL. Additionally, to avoid trivial aggregation of multi-modal features, we introduce contrastive-based regularization to impose additional constraints on the latent representation space. The experimental results demonstrate the effectiveness of FedMAC across various client configurations with statistical heterogeneity, outperforming baseline methods by up to 26% in severe missing scenarios, highlighting its potential as a solution for the challenge of partially missing modalities in federated systems. Our source code is provided at https://github.com/nmduonggg/PEPSY
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。