arXiv:2503.00569cs.LGcs.DC2025-03中稿 · IEEE/ACM Transacti…被引 5

提出一种无需信道分布知识的高效联邦学习设备调度方法。

Communication-Efficient Device Scheduling for Federated Learning Using Lyapunov Optimization

  • 基于李雅普诺夫框架,按参与概率倒数加权更新以加速收敛。
  • 在非独立同分布数据下仍可实现线性加速,通信时间显著降低。
  • 适合无线环境受限、设备异构的联邦学习场景,无需提前知晓信道统计特性。

联邦学习(FL)可在不集中收集数据的前提下实现分布式模型训练。但在受限无线环境中,设备间歇连接、连接质量异构及非独立同分布(non-i.i.d.)数据会严重拖慢收敛速度。本文考虑每轮设备具有任意参与概率的情况,证明通过将各设备更新按其每轮参与概率的倒数加权,可保证收敛至平稳点。该界适用于非凸损失函数与非i.i.d.数据集,并在全参与和均匀部分参与情形下恢复现有最优收敛速率,且仅需单侧学习率。基于此收敛界,我们设计了一种新型在线客户端选择与功率分配算法,利用李雅普诺夫漂移-惩罚框架,在发射功率约束下,动态最小化收敛界与平均通信时间之和。采用流形优化技术求解该问题。得益于李雅普诺夫框架,算法仅需瞬时信道状态信息,无需已知信道分布。在不同数据异构度下的CIFAR-10数据集上仿真表明,相比随机参与策略,本方法在异构信道条件下可显著减少通信时间。

原文摘要 · Abstract (English)

Federated learning (FL) is a useful tool that enables the training of machine learning models over distributed data without having to collect data centrally. When deploying FL in constrained wireless environments, however, intermittent connectivity of devices, heterogeneous connection quality, and non-i.i.d. data can severely slow convergence. In this paper, we consider FL with arbitrary device participation probabilities for each round and show that by weighing each device's update by the reciprocal of their per-round participation probability, we can guarantee convergence to a stationary point. Our bound applies to non-convex loss functions and non-i.i.d. datasets and recovers state-of-the-art convergence rates for both full and uniform partial participation, including linear speedup, with only a single-sided learning rate. Then, using the derived convergence bound, we develop a new online client selection and power allocation algorithm that utilizes the Lyapunov drift-plus-penalty framework to opportunistically minimize a function of the convergence bound and the average communication time under a transmit power constraint. We use optimization over manifold techniques to obtain a solution to the minimization problem. Thanks to the Lyapunov framework, one key feature of the algorithm is that knowledge of the channel distribution is not required and only the instantaneous channel state information needs to be known. Using the CIFAR-10 dataset with varying levels of data heterogeneity, we show through simulations that the communication time can be significantly decreased using our algorithm compared to uniformly random participation, especially for heterogeneous channel conditions.

联邦学习设备调度李雅普诺夫通信效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。