让脉冲神经网络在线学习突触延迟,提升时序识别精度
Three factor delay learning rules for spiking neural networks
- 引入突触与轴突延迟,通过三因子规则同步学习权重与延迟参数
- 在相同参数量下,联合学习使准确率提升14%,最高比仅学权重高20%
- 适合资源受限的类脑芯片,支持设备端实时学习,模型小6.6倍
脉冲神经网络(SNN)处理时空数据,但传统方法仅能学习突触权重,难以捕捉时间模式。本文在基于漏积分-放电(LIF)的前馈与循环SNN中引入突触和轴突延迟,并提出三因子学习规则,实现参数的在线联合学习。采用平滑高斯替代函数仅用于可塑性迹计算,结合自顶向下误差信号更新参数。实验表明,加入延迟后准确率最高较仅学权重基线提升20%;在参数量相近情况下,联合学习使准确率最高提升14%。在SHD语音识别数据集上,本方法达到与离线反向传播相当的精度,相比当前最优方法,模型规模缩小6.6倍,推理延迟降低67%,仅损失2.4%准确率。研究成果有助于设计低功耗、低面积的类脑处理器,支持本地学习并减少内存需求。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) are dynamical systems that operate on spatiotemporal data, yet their learnable parameters are often limited to synaptic weights, contributing little to temporal pattern recognition. Learnable parameters that delay spike times can improve classification performance in temporal tasks, but existing methods rely on large networks and offline learning, making them unsuitable for real-time operation in resource-constrained environments. In this paper, we introduce synaptic and axonal delays to leaky integrate and fire (LIF)-based feedforward and recurrent SNNs, and propose three-factor learning rules to simultaneously learn delay parameters online. We employ a smooth Gaussian surrogate to approximate spike derivatives exclusively for the eligibility trace calculation, and together with a top-down error signal determine parameter updates. Our experiments show that incorporating delays improves accuracy by up to 20% over a weights-only baseline, and for networks with similar parameter counts, jointly learning weights and delays yields up to 14% higher accuracy. On the SHD speech recognition dataset, our method achieves similar accuracy to offline backpropagation-based approaches. Compared to state-of-the-art methods, it reduces model size by 6.6x and inference latency by 67%, with only a 2.4% drop in classification accuracy. Our findings benefit the design of power and area-constrained neuromorphic processors by enabling on-device learning and lowering memory requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。