分析联邦平均算法的偏差来源，并提出新方法降低误差。
Refined Analysis of Federated Averaging and Federated Richardson-Romberg
- 利用马尔可夫过程分析联邦平均收敛性，揭示偏差成因。
- 发现偏差由随机梯度噪声和客户端异构性共同导致。
- 基于理查森-罗姆伯格外推法，有效降低算法偏差，适合分布式学习研究者。
本文针对使用固定步长的联邦平均算法（FedAvg）提出一种新分析方法，基于底层过程的马尔可夫性质，证明全局迭代点收敛至稳态分布，并分析其相对于问题解的偏差与方差。在同质与异质设置下，首次给出一阶偏差展开式。有趣的是，该偏差可分解为两个独立成分：一个仅依赖于随机梯度噪声，另一个源于客户端异构性。最后，我们引入一种基于理查森-罗姆伯格外推技术的新算法，以缓解此偏差问题。
原文摘要 · Abstract (English)
In this paper, we present a novel analysis of \FedAvg with constant step size, relying on the Markov property of the underlying process. We demonstrate that the global iterates of the algorithm converge to a stationary distribution and analyze its resulting bias and variance relative to the problem's solution. We provide a first-order bias expansion in both homogeneous and heterogeneous settings. Interestingly, this bias decomposes into two distinct components: one that depends solely on stochastic gradient noise and another on client heterogeneity. Finally, we introduce a new algorithm based on the Richardson-Romberg extrapolation technique to mitigate this bias.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。