提出ILIF神经元模型,解决脉冲网络过激活问题。
ILIF: Temporal Inhibitory Leaky Integrate-and-Fire Neuron for Overactivation in Spiking Neural Networks
- 引入抑制性机制的时序脉冲神经元,缓解过激活
- 实验显示降低脉冲频率,提升训练稳定性和准确率
- 适合追求能效与稳定性的脉冲神经网络研究者
脉冲神经网络(SNN)因其低功耗、事件驱动和生物合理性受到关注。为通过反向传播训练SNN,需使用代理梯度近似不可微的脉冲函数,但其仅在膜电位接近阈值的窄范围内具有非零导数,称为代理梯度支持宽度γ。我们识别出γ的两难困境:较大γ导致过度激活(过多神经元放电),增加能耗;较小γ则引发梯度消失,削弱时序依赖。为此,我们提出受生物抑制机制启发的时序抑制型漏斗积分-放电(ILIF)神经元模型,通过膜电位与电流间的互抑制结构,有效缓解过激活并维持梯度传播。理论分析证明了其对γ困境的缓解效果,多数据集实验表明,ILIF可降低脉冲率,提升能效,稳定训练并增强精度。代码已开源。
原文摘要 · Abstract (English)
The Spiking Neural Network (SNN) has drawn increasing attention for its energy-efficient, event-driven processing and biological plausibility. To train SNNs via backpropagation, surrogate gradients are used to approximate the non-differentiable spike function, but they only maintain nonzero derivatives within a narrow range of membrane potentials near the firing threshold, referred to as the surrogate gradient support width gamma. We identify a major challenge, termed the dilemma of gamma: a relatively large gamma leads to overactivation, characterized by excessive neuron firing, which in turn increases energy consumption, whereas a small gamma causes vanishing gradients and weakens temporal dependencies. To address this, we propose a temporal Inhibitory Leaky Integrate-and-Fire (ILIF) neuron model, inspired by biological inhibitory mechanisms. This model incorporates interconnected inhibitory units for membrane potential and current, effectively mitigating overactivation while preserving gradient propagation. Theoretical analysis demonstrates ILIF effectiveness in overcoming the gamma dilemma, and extensive experiments on multiple datasets show that ILIF improves energy efficiency by reducing firing rates, stabilizes training, and enhances accuracy. The code is available at github.com/kaisun1/ILIF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。