arXiv:2605.27412cs.NEcs.AI2026-05被引 1

通过动态膜电位与可学习梯度,提升脉冲神经网络直接训练性能。

Advancing Direct Training for Spiking Neural Networks with Circulate-Firing Neurons and Learnable Gradients

论文配图:Advancing Direct Training for Spiking Neural Networks with Circulate-Firing Neurons and Learnable Gradients
图 1 · 摘自论文原文
  • 引入循环放电神经元模型,增强膜电位信息表达能力。
  • 设计逐时步可学习的替代梯度,提升反向传播精度。
  • 适合追求能效与高性能的脉冲神经网络研究者。

脉冲神经网络(SNNs)具备出色的能效优势,但其性能仍显著落后于人工神经网络(ANNs)。这主要源于两大限制:一是传统脉冲神经元的信息表征能力有限,未能充分利用膜电位的丰富动态;二是固定的时间步替代梯度导致梯度传播不准确,阻碍有效直接训练。为此,我们提出一种新直接训练算法,包含三项核心创新:第一,提出循环放电脉冲神经元模型,更有效地利用膜电位动态以增强信息表达能力;第二,设计逐时步可学习的替代梯度函数,实现反向传播中更精准的梯度估计;第三,采用正负平衡损失函数,使正负膜电位趋于均衡,进一步提升性能。大量实验表明,该方法在多个数据集上表现优异,且可无缝推广至Transformer等先进架构,持续优于现有方法。本工作揭示了深入挖掘SNN固有膜电位动态对性能提升的有效性,为构建高性能脉冲神经网络开辟新路径。

原文摘要 · Abstract (English)

Spiking Neural Networks (SNNs) have emerged with promising energy-efficient property, yet a substantial performance gap persists compared to Artificial Neural Networks (ANNs). This gap stems from at least two key limitations: first, conventional spiking neurons offer limited information representation capacity, underutilizing the rich dynamics of membrane potentials; second, fixed surrogate gradient (SG) functions across time steps leads to imprecise gradient propagation, impeding effective direct training. To address these two challenges, we propose a new direct training algorithm with three core innovations: first, a circulate-firing spiking neuron model that enhances information representation capacity by leveraging membrane potentials more effectively; second, a time-step-wise learnable surrogate gradient function, enabling accurate gradient estimation during backpropagation; third, a positive-negative balanced loss function to achieve equilibrium between positive and negative membrane potentials and further boost SNN performance. Extensive experiments demonstrate that our methods achieve competitive performance across multiple datasets. Our methods can generalize seamlessly to advanced architectures of Transformer, consistently outperforming existing methods. Our work highlights the effectiveness of further harnessing intrinsic membrane dynamics of SNNs for performance improvement, and thus open a new avenue for advancing high-performance spiking neural architectures.

脉冲神经网络直接训练可学习梯度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。