通过历史梯度方向投影，显著降低联邦学习通信开销。
Gradient Projection onto Historical Descent Directions for Communication-Efficient Federated Learning
- 将本地梯度投影到历史下降方向构成的共享子空间中
- 在多种场景下收敛性有保障，通信量大幅减少
- 适合大规模模型且对压缩器类型不敏感
联邦学习（FL）可在多个客户端上实现去中心化模型训练，同时可选地保护数据隐私。然而，通信效率仍是大规模模型下的关键瓶颈。本文提出两种互补算法：适用于无偏压缩器的ProjFL，以及通过误差反馈机制适配有偏压缩器的ProjFL+EF。两者均基于将本地梯度投影到由历史下降方向张成的共享客户端-服务器子空间，实现高效信息交换并保持极低通信开销。我们在标准联邦分类基准上对深度神经网络进行实验，结果表明，ProjFL与ProjFL+EF在保持与现有基线相当精度的同时，显著降低了通信成本。收敛性分析覆盖强凸、凸及非凸设置。
原文摘要 · Abstract (English)
Federated Learning (FL) enables decentralized model training across multiple clients while optionally preserving data privacy. However, communication efficiency remains a critical bottleneck, particularly for large-scale models. In this work, we introduce two complementary algorithms: ProjFL, designed for unbiased compressors, and ProjFL+EF, tailored for biased compressors through an Error Feedback mechanism. Both methods rely on projecting local gradients onto a shared client-server subspace spanned by historical descent directions, enabling efficient information exchange with minimal communication overhead. We establish convergence guarantees for both algorithms under strongly convex, convex, and non-convex settings. Empirical evaluations on standard FL classification benchmarks with deep neural networks show that ProjFL and ProjFL+EF achieve accuracy comparable to existing baselines while substantially reducing communication costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。