提出一种通信高效的联邦学习方法,让客户端逐比特更新模型参数。
FedBiF: Communication-Efficient Federated Learning via Bits Freezing
- 训练时直接学习量化参数,客户端每轮只更新1位比特
- 仅用1比特/参数上传,精度仍接近标准联邦平均
- 适合资源受限设备,支持稀疏模型压缩
联邦学习(FL)是一种分布式机器学习范式,可在不共享本地数据的情况下协同训练模型。尽管具有优势,但其通信开销较大,影响训练效率。现有方法通过量化模型更新来降低通信成本,但多数在本地训练后才进行量化,引入量化误差并可能降低模型精度。本文提出联邦比特冻结(FedBiF),一种新型联邦学习框架,在本地训练阶段直接学习量化参数。每轮通信中,服务器先对模型参数进行量化并发送给客户端。随后,每个客户端仅更新多比特参数表示中的一个比特,其余比特保持冻结。这种逐比特更新策略将每次参数更新压缩至1比特,同时保持高精度的参数表示。在五个常用数据集上,分别在IID和Non-IID设置下进行了大量实验。结果表明,FedBiF不仅实现了卓越的通信压缩效果,还促进了模型稀疏性。值得注意的是,即使上行通信仅使用1比特/参数(bpp),下行通信为3 bpp,其性能仍可媲美FedAvg。代码已公开于 https://github.com/Leopold1423/fedbif-tpds25。
原文摘要 · Abstract (English)
Federated learning (FL) is an emerging distributed machine learning paradigm that enables collaborative model training without sharing local data. Despite its advantages, FL suffers from substantial communication overhead, which can affect training efficiency. Recent efforts have mitigated this issue by quantizing model updates to reduce communication costs. However, most existing methods apply quantization only after local training, introducing quantization errors into the trained parameters and potentially degrading model accuracy. In this paper, we propose Federated Bit Freezing (FedBiF), a novel FL framework that directly learns quantized model parameters during local training. In each communication round, the server first quantizes the model parameters and transmits them to the clients. FedBiF then allows each client to update only a single bit of the multi-bit parameter representation, freezing the remaining bits. This bit-by-bit update strategy reduces each parameter update to one bit while maintaining high precision in parameter representation. Extensive experiments are conducted on five widely used datasets under both IID and Non-IID settings. The results demonstrate that FedBiF not only achieves superior communication compression but also promotes sparsity in the resulting models. Notably, FedBiF attains accuracy comparable to FedAvg, even when using only 1 bit-per-parameter (bpp) for uplink and 3 bpp for downlink communication. The code is available at https://github.com/Leopold1423/fedbif-tpds25.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。