解决脉冲神经网络直接训练中的梯度不稳与初始状态偏差问题
Stabilizing Direct Training of Spiking Neural Networks: Membrane Potential Initialization and Threshold-robust Surrogate Gradient
- 通过膜电位初始化对齐稳定分布，缓解时间协变量偏移
- 提出抗阈值扰动的代理梯度，稳定训练过程中的梯度流动
- 在静态与动态图像数据集上达到当前最优精度，适合低功耗神经网络研究者
脉冲神经网络（SNN）的直接训练近年来取得显著进展，即使在早期时间步也能实现高质量输出，为新型节能人工智能范式铺平道路。然而，SNN固有的非线性与时间依赖性带来了持续挑战，如时间协变量偏移（TCS）和可学习神经元阈值下的不稳定梯度流动。本文提出两项关键创新：MP-Init（膜电位初始化）与TrSG（阈值鲁棒代理梯度）。MP-Init通过将初始膜电位与稳态分布对齐来缓解TCS，而TrSG则在训练过程中稳定针对阈值电压的梯度流动。大量实验验证了该方法的有效性，在静态与动态图像数据集上均达到当前最优准确率。代码已开源：https://github.com/kookhh0827/SNN-MP-Init-TRSG。
原文摘要 · Abstract (English)
Recent advancements in the direct training of Spiking Neural Networks (SNNs) have demonstrated high-quality outputs even at early timesteps, paving the way for novel energy-efficient AI paradigms. However, the inherent non-linearity and temporal dependencies in SNNs introduce persistent challenges, such as temporal covariate shift (TCS) and unstable gradient flow with learnable neuron thresholds. In this paper, we present two key innovations: MP-Init (Membrane Potential Initialization) and TrSG (Threshold-robust Surrogate Gradient). MP-Init addresses TCS by aligning the initial membrane potential with its stationary distribution, while TrSG stabilizes gradient flow with respect to threshold voltage during training. Extensive experiments validate our approach, achieving state-of-the-art accuracy on both static and dynamic image datasets. The code is available at: https://github.com/kookhh0827/SNN-MP-Init-TRSG
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。