提出自适应梯度调节机制，解决二值脉冲网络训练中的权重频繁翻转问题。
Towards Accurate Binary Spiking Neural Networks: Learning with Adaptive Gradient Modulation Mechanism
- 设计动态调整梯度的自适应机制，抑制权重符号频繁翻转。
- 在静态与神经形态数据集上实现更快收敛和更高准确率。
- 适合资源受限设备部署，显著提升二值脉冲网络实用性。
二值脉冲神经网络（BSNNs）继承了脉冲神经网络（SNNs）的事件驱动特性，并结合二值化技术降低存储开销，具备轻量化与低功耗优势，适用于资源受限的边缘设备。然而，由于突触权重为二值且脉冲函数不可导，有效训练BSNNs仍是未解难题。本文深入分析了训练中常见的权重符号频繁翻转问题，提出自适应梯度调节机制（AGMM），通过在学习过程中自适应调整梯度，减少权重符号翻转频率。实验表明，该方法可显著加快收敛速度并提升精度，缩小与全精度模型的差距。AGMM在静态与神经形态数据集上均取得当前最优性能，大幅降低存储需求，增强SNN固有的能效优势，使其在资源受限环境中更具可行性。
原文摘要 · Abstract (English)
Binary Spiking Neural Networks (BSNNs) inherit the eventdriven paradigm of SNNs, while also adopting the reduced storage burden of binarization techniques. These distinct advantages grant BSNNs lightweight and energy-efficient characteristics, rendering them ideal for deployment on resource-constrained edge devices. However, due to the binary synaptic weights and non-differentiable spike function, effectively training BSNNs remains an open question. In this paper, we conduct an in-depth analysis of the challenge for BSNN learning, namely the frequent weight sign flipping problem. To mitigate this issue, we propose an Adaptive Gradient Modulation Mechanism (AGMM), which is designed to reduce the frequency of weight sign flipping by adaptively adjusting the gradients during the learning process. The proposed AGMM can enable BSNNs to achieve faster convergence speed and higher accuracy, effectively narrowing the gap between BSNNs and their full-precision equivalents. We validate AGMM on both static and neuromorphic datasets, and results indicate that it achieves state-of-the-art results among BSNNs. This work substantially reduces storage demands and enhances SNNs' inherent energy efficiency, making them highly feasible for resource-constrained environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。