揭示联邦学习非独立同分布数据下性能下降的根源
Understanding Federated Learning from IID to Non-IID dataset: An Experimental Study
- 从梯度下降到联邦学习，分析数据分布对模型影响
- 发现客户端损失曲面不一致是导致性能下降主因
- 提出两类应对策略，为后续研究提供清晰方向
随着隐私担忧和数据监管日益严格，联邦学习（FL）作为一种可在去中心化数据源上训练模型而不共享原始数据的有前景方法应运而生。然而，客户端数据通常是非独立同分布（non-IID）的，导致其性能低于集中式学习。尽管已有多种方法被提出解决此问题，但其内在机制常被不同视角看待。通过从梯度下降到联邦学习、从IID到非IID数据设置的全面研究，我们发现客户端损失曲面的不一致性是造成非IID场景下性能下降的主要原因。基于此理解，我们观察到现有方法可归为两类策略：(i) 调整参数更新路径；(ii) 修改客户端损失曲面。这些发现为应对联邦学习中的非IID挑战提供了清晰视角，并有助于引导该领域的未来研究。
原文摘要 · Abstract (English)
As privacy concerns and data regulations grow, federated learning (FL) has emerged as a promising approach for training machine learning models across decentralized data sources without sharing raw data. However, a significant challenge in FL is that client data are often non-IID (non-independent and identically distributed), leading to reduced performance compared to centralized learning. While many methods have been proposed to address this issue, their underlying mechanisms are often viewed from different perspectives. Through a comprehensive investigation from gradient descent to FL, and from IID to non-IID data settings, we find that inconsistencies in client loss landscapes primarily cause performance degradation in non-IID scenarios. From this understanding, we observe that existing methods can be grouped into two main strategies: (i) adjusting parameter update paths and (ii) modifying client loss landscapes. These findings offer a clear perspective on addressing non-IID challenges in FL and help guide future research in the field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。