arXiv:2512.22623cs.LGeess.SP2025-12

提出两种无需客户端状态的压缩通信方法,提升联邦学习效率。

Communication Compression for Distributed Learning with Aggregate and Server-Guided Feedback

  • 用全局聚合更新作共享控制变量,避免客户端存状态
  • 服务器有少量数据时可生成更准预测更新,加速收敛
  • 理论证明优于现有压缩方法,适合大规模隐私敏感场景

分布式学习,特别是联邦学习(FL),在通信成本上面临显著瓶颈,尤其是受限于边缘侧不对称带宽的客户端到服务器更新上传。有偏压缩技术在实践中有效,但需误差反馈机制以保证理论收敛性。标准误差反馈依赖客户端特定的控制变量,违反用户隐私且不兼容大规模联邦学习中常见的无状态客户端。本文提出两种新框架:第一种为压缩聚合反馈(CAFe),使用前一轮全局聚合更新作为所有客户端的共享控制变量;第二种为服务器引导的压缩聚合反馈(CAFe-S),适用于服务器拥有少量私有数据的情况,生成更精确的候选更新。以分布式梯度下降(DGD)为例,我们分析证明了在非凸情形下、梯度差异有界时,CAFe优于带偏压缩的分布式压缩梯度下降(DCGD)。进一步证明,CAFe-S能收敛至稳定点,且收敛速度随服务器数据代表性增强而提高。实验结果在联邦学习场景中验证了本方法对现有压缩方案的优越性。

原文摘要 · Abstract (English)

Distributed learning, particularly Federated Learning (FL), faces a significant bottleneck in the communication cost, particularly the uplink transmission of client-to-server updates, which is often constrained by asymmetric bandwidth limits at the edge. Biased compression techniques are effective in practice, but require error feedback mechanisms to provide theoretical guarantees and to ensure convergence when compression is aggressive. Standard error feedback, however, relies on client-specific control variates, which violates user privacy and is incompatible with stateless clients common in large-scale FL. This paper proposes two novel frameworks that enable biased compression without client-side state or control variates. The first, Compressed Aggregate Feedback (CAFe), uses the globally aggregated update from the previous round as a shared control variate for all clients. The second, Server-Guided Compressed Aggregate Feedback (CAFe-S), extends this idea to scenarios where the server possesses a small private dataset; it generates a server-guided candidate update to be used as a more accurate predictor. We consider Distributed Gradient Descent (DGD) as a representative algorithm and analytically prove CAFe's superiority to Distributed Compressed Gradient Descent (DCGD) with biased compression in the non-convex regime with bounded gradient dissimilarity. We further prove that CAFe-S converges to a stationary point, with a rate that improves as the server's data become more representative. Experimental results in FL scenarios validate the superiority of our approaches over existing compression schemes.

联邦学习通信压缩隐私保护分布式优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。