大规模联邦学习用无线信号传输参数,反而提升隐私与收敛速度。
Rethinking Federated Learning Over the Air: The Blessing of Scaling Up
- 用模拟信号无线传输中间参数,支持更多客户端并发通信。
- 客户端越多,隐私泄露越少,噪声和信道衰落影响越小。
- 适合高密度终端的隐私保护型联合训练场景。
联邦学习可在保护数据隐私的前提下实现多客户端协同建模,但受限于通信资源,尤其在大量客户端参与时性能下降。将空中计算引入训练流程,通过模拟信号传输中间参数,显著提升每轮通信可支持的客户端数量。然而,这会引入由信道衰落和噪声引起的失真。本文建立理论框架分析大规模场景下空中联邦学习的性能,揭示三个关键优势:(1)客户端数量增加使本地梯度与服务器聚合梯度之间的互信息降低,有效减少隐私泄露;(2)信道硬化效应消除小尺度衰落对噪声梯度的影响;(3)热噪声和梯度估计误差减小,加快模型收敛速度。理论分析经大量实验验证,证实空中联邦学习在大规模网络中具备可行性。
原文摘要 · Abstract (English)
Federated learning facilitates collaborative model training across multiple clients while preserving data privacy. However, its performance is often constrained by limited communication resources, particularly in systems supporting a large number of clients. To address this challenge, integrating over-the-air computations into the training process has emerged as a promising solution to alleviate communication bottlenecks. The system significantly increases the number of clients it can support in each communication round by transmitting intermediate parameters via analog signals rather than digital ones. This improvement, however, comes at the cost of channel-induced distortions, such as fading and noise, which affect the aggregated global parameters. To elucidate these effects, this paper develops a theoretical framework to analyze the performance of over-the-air federated learning in large-scale client scenarios. Our analysis reveals three key advantages of scaling up the number of participating clients: (1) Enhanced Privacy: The mutual information between a client's local gradient and the server's aggregated gradient diminishes, effectively reducing privacy leakage. (2) Mitigation of Channel Fading: The channel hardening effect eliminates the impact of small-scale fading in the noisy global gradient. (3) Improved Convergence: Reduced thermal noise and gradient estimation errors benefit the convergence rate. These findings solidify over-the-air model training as a viable approach for federated learning in networks with a large number of clients. The theoretical insights are further substantiated through extensive experimental evaluations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。