利用脉冲神经网络的抗噪特性，实现联邦学习中高效低带宽通信。
The Robustness of Spiking Neural Networks in Communication and its Application towards Network Efficiency in Federated Learning
- 提出基于顶K稀疏化的联邦学习算法，降低通信数据量。
- 通信参数量可压缩至原模型的6%，且不损失精度。
- 适合资源受限设备的高效联邦学习应用。
脉冲神经网络（SNNs）因其在嵌入式设备上的低功耗优势，成为传统人工神经网络（ANNs）的节能替代方案。然而，在涉及多方协作训练的联邦学习（FL）场景中，设备与服务器间的通信仍构成瓶颈，且成本高昂。本文首次研究了SNN在噪声通信环境下的固有鲁棒性，并在此基础上提出一种新型的顶K稀疏化联邦学习算法（FLTS），以减少训练过程中的带宽消耗。实验表明，相比ANNs，采用SNN的该方案能实现更高的带宽节省，同时保持模型精度。通信参数量可降至原始模型的6%。此外，通过在训练过程中动态压缩参数，进一步提升了通信效率。大量实验结果证明，所提算法在通信开销和模型精度上均显著优于基线方法，为基于SNN的实用化高效联邦学习提供了可行路径。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) have recently gained significant interest in on-chip learning in embedded devices and emerged as an energy-efficient alternative to conventional Artificial Neural Networks (ANNs). However, to extend SNNs to a Federated Learning (FL) setting involving collaborative model training, the communication between the local devices and the remote server remains the bottleneck, which is often restricted and costly. In this paper, we first explore the inherent robustness of SNNs under noisy communication in FL. Building upon this foundation, we propose a novel Federated Learning with Top-K Sparsification (FLTS) algorithm to reduce the bandwidth usage for FL training. We discover that the proposed scheme with SNNs allows more bandwidth savings compared to ANNs without impacting the model's accuracy. Additionally, the number of parameters to be communicated can be reduced to as low as 6 percent of the size of the original model. We further improve the communication efficiency by enabling dynamic parameter compression during model training. Extensive experiment results demonstrate that our proposed algorithms significantly outperform the baselines in terms of communication cost and model accuracy and are promising for practical network-efficient FL with SNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。