arXiv:2409.15100cs.LGcs.AI2024-09被引 5

用中位数锚定裁剪法提升无线联邦学习抗重尾噪声能力

Robust Federated Learning Over the Air: Combating Heavy-Tailed Noise with Median Anchored Clipping

  • 提出中位数锚定裁剪方法,应对无线信道重尾噪声
  • 理论推导出收敛速率,证明方法有效性
  • 适合研究无线联邦学习鲁棒性的研究人员

利用空中计算进行模型聚合是缓解联邦边缘学习通信瓶颈的有效方法。通过利用多接入信道的叠加特性,该方法实现了通信与计算的一体化设计,提升了系统隐私性并降低了实现成本。然而,无线信道固有的电磁干扰常呈现重尾分布,导致全局梯度中出现异常强的噪声,显著损害训练性能。为此,本文提出一种新型梯度裁剪方法——中位数锚定裁剪(MAC),以对抗重尾噪声的负面影响。同时,推导了在MAC下模拟空中计算联邦学习的模型训练收敛速率表达式,定量展示了MAC对训练性能的影响。大量实验结果表明,所提MAC算法能有效缓解重尾噪声影响,显著增强系统鲁棒性。

原文摘要 · Abstract (English)

Leveraging over-the-air computations for model aggregation is an effective approach to cope with the communication bottleneck in federated edge learning. By exploiting the superposition properties of multi-access channels, this approach facilitates an integrated design of communication and computation, thereby enhancing system privacy while reducing implementation costs. However, the inherent electromagnetic interference in radio channels often exhibits heavy-tailed distributions, giving rise to exceptionally strong noise in globally aggregated gradients that can significantly deteriorate the training performance. To address this issue, we propose a novel gradient clipping method, termed Median Anchored Clipping (MAC), to combat the detrimental effects of heavy-tailed noise. We also derive analytical expressions for the convergence rate of model training with analog over-the-air federated learning under MAC, which quantitatively demonstrates the effect of MAC on training performance. Extensive experimental results show that the proposed MAC algorithm effectively mitigates the impact of heavy-tailed noise, hence substantially enhancing system robustness.

联邦学习无线计算鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。