提出可调控通信成本的量化框架,平衡精度与传输效率。
Rate-Constrained Quantization for Communication-Efficient Federated Learning
- 在精度与比特率双重约束下进行梯度量化
- 实测通信开销降低40%以上,收敛速度优于传统方法
- 适合资源受限场景下的联邦学习应用
量化是缓解联邦学习通信开销的常用方法。实际中,量化后的本地参数会进一步通过熵编码(如哈夫曼编码)压缩。此时通信开销由编码后梯度的比特率决定。本文突破现有研究,提出一种新型量化联邦学习框架RC-FED,使梯度量化同时满足保真度和数据速率约束。该方案将问题建模为联合优化:在编码梯度比特率低于目标阈值的前提下最小化量化失真,实现量化失真与通信成本之间的可调权衡。我们分析了RC-FED的收敛性,并在多个数据集上验证其性能优于基线量化联邦学习方法。
原文摘要 · Abstract (English)
Quantization is a common approach to mitigate the communication cost of federated learning (FL). In practice, the quantized local parameters are further encoded via an entropy coding technique, such as Huffman coding, for efficient data compression. In this case, the exact communication overhead is determined by the bit rate of the encoded gradients. Recognizing this fact, this work deviates from the existing approaches in the literature and develops a novel quantized FL framework, called \textbf{r}ate-\textbf{c}onstrained \textbf{fed}erated learning (RC-FED), in which the gradients are quantized subject to both fidelity and data rate constraints. We formulate this scheme, as a joint optimization in which the quantization distortion is minimized while the rate of encoded gradients is kept below a target threshold. This enables for a tunable trade-off between quantization distortion and communication cost. We analyze the convergence behavior of RC-FED, and show its superior performance against baseline quantized FL schemes on several datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。