提出首个可用于深度脉冲网络的非线性脉冲神经元离散模型,提升能效与表达力。
Discretized Quadratic Integrate-and-Fire Neuron Model for Deep Spiking Neural Networks
- 将非线性脉冲神经元离散化,实现高效计算与稳定训练
- 在CIFAR-10/100、ImageNet等数据集上超越主流线性模型
- 适合追求高能效与强动态建模能力的脉冲神经网络研究者
脉冲神经网络(SNNs)因其异步、类脑的神经元动力学,成为低功耗替代传统神经网络的有力候选。现有模型中,漏电积分-发放(LIF)神经元因结构简单、计算高效而被广泛采用,但其线性衰减机制限制了表达能力。相比之下,二次积分-发放(QIF)神经元具有更丰富的非线性动力学,但因训练不稳而应用受限。本文首次提出适用于深度SNN的QIF神经元离散化模型,并推导出基于参数集的解析代理梯度窗口,有效缓解梯度失配问题。在CIFAR-10、CIFAR-100、ImageNet及CIFAR-10 DVS数据集上的实验表明,该方法显著优于当前主流的LIF基模型,验证了其在保持可扩展性的同时,兼具更强动态表达能力的潜力。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) have emerged as energy-efficient alternatives to traditional artificial neural networks, leveraging asynchronous and biologically inspired neuron dynamics. Among existing neuron models, the Leaky Integrate-and-Fire (LIF) neuron has become widely adopted in deep SNNs due to its simplicity and computational efficiency. However, this efficiency comes at the expense of expressiveness, as LIF dynamics are constrained to linear decay at each timestep. In contrast, more complex models, such as the Quadratic Integrate-and-Fire (QIF) neuron, exhibit richer, nonlinear dynamics but have seen limited adoption due to their training instability. On that note, we propose the first discretization of the QIF neuron model tailored for high-performance deep spiking neural networks and provide an in-depth analysis of its dynamics. To ensure training stability, we derive an analytical formulation for surrogate gradient windows directly from our discretizations' parameter set, minimizing gradient mismatch. We evaluate our method on CIFAR-10, CIFAR-100, ImageNet, and CIFAR-10 DVS, demonstrating its ability to outperform state-of-the-art LIF-based methods. These results establish our discretization of the QIF neuron as a compelling alternative to LIF neurons for deep SNNs, combining richer dynamics with practical scalability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。