改进联邦学习通信效率，让模型训练更快更稳。
SA-PEF: Step-Ahead Partial Error Feedback for Efficient Federated Learning
- 结合前向修正与部分误差反馈，动态调节梯度更新
- 在非独立同分布数据下仍能快速收敛，比传统方法快约15%-30%
- 适合大规模分布式训练，尤其数据异构场景
带有误差反馈（EF）的有偏梯度压缩可减少联邦学习（FL）中的通信开销，但在非独立同分布（non-IID）数据下，残差误差衰减缓慢，导致梯度不匹配和早期训练停滞。本文提出步前部分误差反馈（SA-PEF），融合步前（SA）修正与部分误差反馈（PEF）。当步前系数 α=0 时退化为标准 EF，α=1 时等价于步前 EF（SAEF）。针对非凸目标函数和 δ-压缩算子，建立了二阶矩界与残差递推关系，证明在数据异构和部分客户端参与下仍可收敛至平稳点，收敛速率与标准非凸 Fed-SGD 一致，达到 $O((η,η_0TR)^{-1})$ 的收敛率，且在固定内层步长下趋于方差/异质性极限。分析揭示了由步前控制的残差收缩率 $ρ_r$，解释了早期训练加速现象。通过选取接近理论最优的 α 值，平衡 SAEF 快速预热与 EF 长期稳定性。跨多种架构与数据集的实验表明，SA-PEF 持续比 EF 更快达到目标精度。
原文摘要 · Abstract (English)
Biased gradient compression with error feedback (EF) reduces communication in federated learning (FL), but under non-IID data, the residual error can decay slowly, causing gradient mismatch and stalled progress in the early rounds. We propose step-ahead partial error feedback (SA-PEF), which integrates step-ahead (SA) correction with partial error feedback (PEF). SA-PEF recovers EF when the step-ahead coefficient $α=0$ and step-ahead EF (SAEF) when $α=1$. For non-convex objectives and $δ$-contractive compressors, we establish a second-moment bound and a residual recursion that guarantee convergence to stationarity under heterogeneous data and partial client participation. The resulting rates match standard non-convex Fed-SGD guarantees up to constant factors, achieving $O((η,η_0TR)^{-1})$ convergence to a variance/heterogeneity floor with a fixed inner step size. Our analysis reveals a step-ahead-controlled residual contraction $ρ_r$ that explains the observed acceleration in the early training phase. To balance SAEF's rapid warm-up with EF's long-term stability, we select $α$ near its theory-predicted optimum. Experiments across diverse architectures and datasets show that SA-PEF consistently reaches target accuracy faster than EF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。