提出分数阶脉冲时序梯度下降法,显著提升脉冲神经网络分类准确率。
Fractional-order spike-timing-dependent gradient descent for multi-layer spiking neural networks
- 引入分数阶梯度下降,结合脉冲时序依赖机制优化学习
- 分数阶1.9时准确率比传统方法提升155%
- 适用于低功耗类脑计算,适合神经形态硬件部署
人类大脑神经活动的丰富知识推动了类脑脉冲神经网络(SNNs)的发展。与非脉冲深度神经网络(DNNs)相比,SNNs能通过生物合理、低功耗的事件驱动架构更高效地编码和传输时空信息。然而,由于突触可塑性在现有反向传播框架中难以实现和解释,SNN的有监督学习仍具挑战。本文提出分数阶脉冲时序依赖梯度下降(FO-STDGD)学习模型,基于非漏电积分-发放神经元的瞬时发放率与膜电位之间的非线性关系推导激活函数。该训练策略可推广至0到2之间的任意分数阶,因将分数阶梯度下降融入脉冲时序损失梯度计算。在MNIST和DVS128 Gesture数据集上的测试表明,随着分数阶增加,分类准确率提高;其中分数阶1.9相较传统梯度下降(阶数1)提升155%。此外,本方案在相同网络结构和训练轮次下实现了当前最优的计算效率。
原文摘要 · Abstract (English)
Accumulated detailed knowledge about the neuronal activities in human brains has brought more attention to bio-inspired spiking neural networks (SNNs). In contrast to non-spiking deep neural networks (DNNs), SNNs can encode and transmit spatiotemporal information more efficiently by exploiting biologically realistic and low-power event-driven neuromorphic architectures. However, the supervised learning of SNNs still remains a challenge because the spike-timing-dependent plasticity (STDP) of connected spiking neurons is difficult to implement and interpret in existing backpropagation learning schemes. This paper proposes a fractional-order spike-timing-dependent gradient descent (FO-STDGD) learning model by considering a derived nonlinear activation function that describes the relationship between the quasi-instantaneous firing rate and the temporal membrane potentials of nonleaky integrate-and-fire neurons. The training strategy can be generalized to any fractional orders between 0 and 2 since the FO-STDGD incorporates the fractional gradient descent method into the calculation of spike-timing-dependent loss gradients. The proposed FO-STDGD model is tested on the MNIST and DVS128 Gesture datasets and its accuracy under different network structure and fractional orders is analyzed. It can be found that the classification accuracy increases as the fractional order increases, and specifically, the case of fractional order 1.9 improves by 155% relative to the case of fractional order 1 (traditional gradient descent). In addition, our scheme demonstrates the state-of-the-art computational efficacy for the same SNN structure and training epochs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。