网络越宽，联邦学习受数据异质性影响越小，无限宽时等同于集中式学习。
Widening the Network Mitigates the Impact of Data Heterogeneity on FedAvg
- 通过理论分析证明：网络宽度增加可削弱数据异质性的影响
- 无限宽时，联邦平均算法的全局与本地模型均退化为线性模型
- 实验验证了多种架构、损失函数和优化器下的理论结论
联邦学习（FL）使分布式客户端在不共享本地数据的情况下协同训练模型。与集中式学习的关键区别在于，客户端数据是非独立同分布的，这对训练出能在异构数据分布上良好泛化的全局模型构成了重大挑战。本文分析了使用梯度下降（GD）的过参数化FedAvg的收敛性。我们证明，随着神经网络宽度增加，数据异质性的影响逐渐减弱，当宽度趋近无穷时，该影响完全消失。在无限宽情况下，我们进一步证明，FedAvg中的全局与局部模型均表现为线性模型，且其泛化性能与相同迭代次数的集中式学习相当。大量实验在多种网络结构、损失函数和优化方法下验证了理论结果。
原文摘要 · Abstract (English)
Federated learning (FL) enables decentralized clients to train a model collaboratively without sharing local data. A key distinction between FL and centralized learning is that clients' data are non-independent and identically distributed, which poses significant challenges in training a global model that generalizes well across heterogeneous local data distributions. In this paper, we analyze the convergence of overparameterized FedAvg with gradient descent (GD). We prove that the impact of data heterogeneity diminishes as the width of neural networks increases, ultimately vanishing when the width approaches infinity. In the infinite-width regime, we further prove that both the global and local models in FedAvg behave as linear models, and that FedAvg achieves the same generalization performance as centralized learning with the same number of GD iterations. Extensive experiments validate our theoretical findings across various network architectures, loss functions, and optimization methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。