提出量化异步联邦学习算法，解决通信瓶颈与误差传播问题。
Quantized and Asynchronous Federated Learning
- 引入隐状态量化机制，避免直接量化导致的误差累积
- 在非凸目标下实现最优的1/√T收敛速率，无需限制梯度或客户端到达规律
- 支持安全聚合等技术，适合大规模分布式训练场景
近期联邦学习进展表明，异步变体比同步版本更快且更具可扩展性。然而，现有设计未包含量化，而量化在实际中对缓解通信瓶颈至关重要。为此，我们提出一种新算法——量化异步联邦学习（QAFeL），采用隐状态量化方案，避免直接量化引发的误差传播。QAFeL还引入缓冲区聚合客户端更新，确保可扩展性并兼容如安全聚合等技术。理论上，我们证明了在非凸目标上的随机梯度下降中，QAFeL达到最优的$/mathcal{O}(1/\ oot{2}{T})$遍历收敛速率，无需有界梯度或均匀客户端到达假设。同时证明，延迟与量化之间的交叉项误差仅影响高阶误差项。我们在标准基准上验证了理论结果。
原文摘要 · Abstract (English)
Recent advances in federated learning have shown that asynchronous variants can be faster and more scalable than their synchronous counterparts. However, their design does not include quantization, which is necessary in practice to deal with the communication bottleneck. To bridge this gap, we develop a novel algorithm, Quantized Asynchronous Federated Learning (QAFeL), which introduces a hidden-state quantization scheme to avoid the error propagation caused by direct quantization. QAFeL also includes a buffer to aggregate client updates, ensuring scalability and compatibility with techniques such as secure aggregation. Furthermore, we prove that QAFeL achieves an $\mathcal{O}(1/\sqrt{T})$ ergodic convergence rate for stochastic gradient descent on non-convex objectives, which is the optimal order of complexity, without requiring bounded gradients or uniform client arrivals. We also prove that the cross-term error between staleness and quantization only affects the higher-order error terms. We validate our theoretical findings on standard benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。