提出FAIR-k算法,让联邦学习通信更高效及时。
Timely Parameter Updating in Over-the-Air Federated Learning
- 按梯度重要性和更新时效性动态选参与上传的参数
- 理论证明算法能加速收敛并减少通信开销
- 适合资源受限的边缘联邦学习场景
将过空气计算(OAC)引入联邦学习(FL)可有效缓解通信瓶颈。在OAC-FL中,各客户端将中间参数(如梯度)调制到同一组正交波形上,同时发送至边缘服务器。利用多址信道的叠加特性,服务器可从接收信号中自动获取全局梯度。然而,实际系统中可用的正交波形数量有限,与现代深度学习模型的高维参数不匹配。为此,我们提出新颖的FAIR-k算法:每轮通信中选择最具影响力的梯度子集进行空中更新。该方法结合轮询与Top-k的优势,在参数更新时效性与重要性间取得平衡。基于马尔可夫分析,我们刻画了参数陈旧性分布,并建立了含FAIR-k的OAC-FL收敛速率。结果揭示数据异质性、信道噪声与参数陈旧性的联合影响。不同于传统假设所有客户端具有统一的Lipschitz常数,本框架采用更精细的数据异质性建模。分析表明,由于促进新鲜且公平的参数更新,FAIR-k不仅加速收敛,还能通过延长本地训练周期提升通信效率,而不会显著影响整体训练效果。
原文摘要 · Abstract (English)
Incorporating over-the-air computations (OAC) into the model training process of federated learning (FL) is an effective approach to alleviating the communication bottleneck in FL systems. Under OAC-FL, every client modulates its intermediate parameters, such as gradient, onto the same set of orthogonal waveforms and simultaneously transmits the radio signal to the edge server. By exploiting the superposition property of multiple-access channels, the edge server can obtain an automatically aggregated global gradient from the received signal. However, the limited number of orthogonal waveforms available in practical systems is fundamentally mismatched with the high dimensionality of modern deep learning models. To address this issue, we propose Freshness Freshness-mAgnItude awaRe top-k (FAIR-k), an algorithm that selects, in each communication round, the most impactful subset of gradients to be updated over the air. In essence, FAIR-k combines the complementary strengths of the Round-Robin and Top-k algorithms, striking a delicate balance between timeliness (freshness of parameter updates) and importance (gradient magnitude). Leveraging tools from Markov analysis, we characterize the distribution of parameter staleness under FAIR-k. Building on this, we establish the convergence rate of OAC-FL with FAIR-k, which discloses the joint effect of data heterogeneity, channel noise, and parameter staleness on the training efficiency. Notably, as opposed to conventional analyses that assume a universal Lipschitz constant across all the clients, our framework adopts a finer-grained model of the data heterogeneity. The analysis demonstrates that since FAIR-k promotes fresh (and fair) parameter updates, it not only accelerates convergence but also enhances communication efficiency by enabling an extended period of local training without significantly affecting overall training efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。