提出双向压缩联邦学习方法，显著降低通信开销。
BICompFL: Stochastic Federated Learning with Bi-Directional Compression
- 采用双向压缩策略，结合重要性采样优化通信效率。
- 实验显示通信成本降低一个数量级，精度仍达顶尖水平。
- 适合资源受限的分布式学习场景，如移动设备协同训练。
针对联邦学习中的显著通信瓶颈，本文研究随机联邦学习（stochastic FL），其中模型或压缩更新由分布而非确定参数定义。该方法在理想下行传输下可有效降低通信负载，但实际中上下行通信均受约束。本文揭示双向压缩在随机联邦学习中的固有挑战，并提出BICompFL解决方案。实验表明，相比多个基准方法，BICompFL将通信成本降低一个数量级，同时保持最先进的准确率。理论分析通过基于重要性采样的新方法，揭示了上下行通信成本之间的相互作用关系。
原文摘要 · Abstract (English)
We address the prominent communication bottleneck in federated learning (FL). We specifically consider stochastic FL, in which models or compressed model updates are specified by distributions rather than deterministic parameters. Stochastic FL offers a principled approach to compression, and has been shown to reduce the communication load under perfect downlink transmission from the federator to the clients. However, in practice, both the uplink and downlink communications are constrained. We show that bi-directional compression for stochastic FL has inherent challenges, which we address by introducing BICompFL. Our BICompFL is experimentally shown to reduce the communication cost by an order of magnitude compared to multiple benchmarks, while maintaining state-of-the-art accuracies. Theoretically, we study the communication cost of BICompFL through a new analysis of an importance-sampling based technique, which exposes the interplay between uplink and downlink communication costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。