解决联邦学习中梯度过时问题,提升训练稳定性与准确率
FedSteer: Taming Extreme Gradient Staleness in Federated Learning with Corrective Projections and Caching

- 用缓存梯度构建低维优化空间,对齐旧梯度方向
- 在极端客户端参与不均下,避免性能崩溃,准确率提升超7%
- 仅缓存代表性客户端,降低服务器内存开销,适合资源受限场景
联邦学习常因客户端参与不一致导致聚合方差。尽管重用离线客户端的旧模型更新可缓解方差,但当客户端参与分布严重偏斜时,更新延迟会变得极其严重,足以破坏训练稳定性。为此,我们提出FedSteer:通过缓存近期客户端梯度构建梯度子空间,作为当前优化景观的低维表征。主动客户端的真梯度被投影到该子空间以获取最优坐标;对非活跃客户端,则复用这些坐标,结合其他活跃客户端演化后的子空间进行调整。这一过程有效将过时梯度“校正”至当前全局目标方向。同时采用选择性缓存策略,仅保留代表性客户端子集构建子空间,减少服务器内存消耗。实验表明,FedSteer显著优于基线,在严苛场景下防止性能崩塌,部分情况下准确率提升超过7%。
原文摘要 · Abstract (English)
Federated learning (FL) is often subject to aggregation variance if clients do not consistently participate in training rounds. While reusing stale model updates from inactive clients is a common technique to reduce this variance, we find that with skewed client participation, the resulting update staleness can become severe enough to destabilize training. To remedy this, we propose FedSteer, a novel method that constructs a gradient subspace from a cache of recent client gradients to serve as a low-dimensional representation of the current optimization landscape. FedSteer projects an active client's true gradient onto this subspace to find a set of optimal coordinates. For an inactive client, FedSteer reuses these coordinates with the now-evolved subspace drifted by other active clients. This process effectively "steers" outdated gradients toward the current global objective. This is complemented by a selective caching strategy that identifies a representative client subset to form the subspace, reducing server memory. Experiments demonstrate that FedSteer significantly outperforms baselines, preventing performance collapse in challenging scenarios while delivering accuracy gains of over 7% in others.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。