解决长尾分布下联邦学习的收敛难题，提升模型公平性与效率
FedWCM: Unleashing the Potential of Momentum-based Federated Learning in Long-Tailed Scenarios
- 动态调整动量，结合全局与每轮数据平衡偏差
- 在长尾数据下实现稳定收敛，准确率显著优于基线方法
- 适合处理数据不平衡的分布式场景，如医疗、金融建模
联邦学习（FL）可在保护数据隐私的前提下实现去中心化模型训练。然而，面对非同分布（non-IID）数据，尤其是类别样本严重失衡的长尾场景时，传统基于动量的联邦学习方法易产生模型偏倚，导致难以收敛。本文通过深入分析神经网络各层行为，揭示了动量机制在长尾分布下的方向性偏差问题。为此，提出FedWCM：一种利用全局及每轮局部数据动态调节动量的策略，有效修正因数据不均引入的方向偏差。大量实验表明，该方法能解决非收敛问题，在多个长尾数据集上显著优于现有方法，显著提升联邦学习在客户端异质性和数据不平衡条件下的效率与效果。
原文摘要 · Abstract (English)
Federated Learning (FL) enables decentralized model training while preserving data privacy. Despite its benefits, FL faces challenges with non-identically distributed (non-IID) data, especially in long-tailed scenarios with imbalanced class samples. Momentum-based FL methods, often used to accelerate FL convergence, struggle with these distributions, resulting in biased models and making FL hard to converge. To understand this challenge, we conduct extensive investigations into this phenomenon, accompanied by a layer-wise analysis of neural network behavior. Based on these insights, we propose FedWCM, a method that dynamically adjusts momentum using global and per-round data to correct directional biases introduced by long-tailed distributions. Extensive experiments show that FedWCM resolves non-convergence issues and outperforms existing methods, enhancing FL's efficiency and effectiveness in handling client heterogeneity and data imbalance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。