提出FedCET算法，在异构数据下实现线性收敛，通信开销更小。
Communication Efficient Federated Learning with Linear Convergence on Heterogeneous Data
- 用可调学习率加权客户端信息，缓解异构数据下的客户端漂移问题。
- 理论证明在合适学习率下可线性收敛至精确解，无需共享梯度和修正项。
- 仅需传输一个变量，通信开销显著降低，适合资源受限场景。
通过让本地客户端在与参数服务器通信前执行多次本地更新，现代联邦学习算法（如FedAvg）有效缓解了分布式学习中的通信瓶颈，并已成功应用于多个场景。然而，这种本地更新与通信之间的异步性在数据异构时会导致‘客户端漂移’问题，影响最终学习结果的准确性。本文提出一种名为FedCET的联邦学习算法，可在客户端数据分布异构条件下确保准确收敛。受分布式优化算法NIDS启发，该方法通过学习率对来自本地客户端的信息进行加权，消除‘客户端漂移’。理论上证明：在适当学习率下，FedCET可实现对精确解的线性收敛。与现有需同时共享梯度和漂移修正项才能保证收敛的算法不同，FedCET仅需共享单一变量，显著降低通信开销。数值实验对比验证了其有效性。
原文摘要 · Abstract (English)
By letting local clients perform multiple local updates before communicating with a parameter server, modern federated learning algorithms such as FedAvg tackle the communication bottleneck problem in distributed learning and have found many successful applications. However, this asynchrony between local updates and communication also leads to a ''client-drift'' problem when the data is heterogeneous (not independent and identically distributed), resulting in errors in the final learning result. In this paper, we propose a federated learning algorithm, which is called FedCET, to ensure accurate convergence even under heterogeneous distributions of data across clients. Inspired by the distributed optimization algorithm NIDS, we use learning rates to weight information received from local clients to eliminate the ''client-drift''. We prove that under appropriate learning rates, FedCET can ensure linear convergence to the exact solution. Different from existing algorithms which have to share both gradients and a drift-correction term to ensure accurate convergence under heterogeneous data distributions, FedCET only shares one variable, which significantly reduces communication overhead. Numerical comparison with existing counterpart algorithms confirms the effectiveness of FedCET.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。