arXiv:2509.13053cs.LGcs.NE2025-09被引 2

提出新型脉冲神经网络学习法,高效省内存,适合边缘设备部署。

Traces Propagation: Memory-Efficient and Scalable Forward-Only Learning in Spiking Neural Networks

  • 用迹传播机制实现全局部前向学习,无需反向传播
  • 在多个数据集上性能优于现有本地学习规则,支持深层网络
  • 内存占用低,适合嵌入式设备上的关键词识别等实际任务

脉冲神经网络(SNN)为处理时空动态信号和研究生物神经系统的学习机制提供了高效框架。训练SNN的关键挑战在于解决空间与时间的信用分配问题。主流方法是基于替代梯度的时序反向传播(BPTT),但其违背了生物系统中时空局部性原则,且计算与内存开销高,限制了高效训练和边缘端学习。现有局部学习规则虽能通过可塑性迹实现局部时间信用分配,却需依赖额外分层矩阵解决空间信用分配,增加内存负担,影响可扩展性。本文提出迹传播(Traces Propagation, TP),一种全前向、内存高效、可扩展且完全局部的学习规则,结合可塑性迹与分层对比损失,无需辅助分层矩阵。TP在NMNIST和SHD数据集上优于其他全局部学习规则;在更复杂的DVS-GESTURE和DVS-CIFAR10数据集上表现竞争力,并可有效拓展至VGG-9等深层架构,相比已有全局部可扩展规则具有更优内存增长特性。此外,我们验证了TP在谷歌语音命令数据集上的关键词检测微调任务中的适用性,为边缘端高效学习铺平道路。

原文摘要 · Abstract (English)

Spiking Neural Networks (SNNs) provide an efficient framework for processing dynamic spatio-temporal signals and for investigating the learning principles underlying biological neural systems. A key challenge in training SNNs is to solve both spatial and temporal credit assignment. The dominant approach for training SNNs is Backpropagation Through Time (BPTT) with surrogate gradients. However, BPTT is in stark contrast with the spatial and temporal locality observed in biological neural systems and leads to high computational and memory demands, limiting efficient training strategies and on-device learning. Although existing local learning rules achieve local temporal credit assignment by leveraging eligibility traces, they fail to address the spatial credit assignment without resorting to auxiliary layer-wise matrices, which increase memory overhead and hinder scalability, especially on embedded devices. In this work, we propose Traces Propagation (TP), a forward-only, memory-efficient, scalable, and fully local learning rule that combines eligibility traces with a layer-wise contrastive loss without requiring auxiliary layer-wise matrices. TP outperforms other fully local learning rules on NMNIST and SHD datasets. On more complex datasets such as DVS-GESTURE and DVS-CIFAR10, TP showcases competitive performance and scales effectively to deeper SNN architectures such as VGG-9, while providing favorable memory scaling compared to prior fully local scalable rules, for datasets with a significant number of classes. Finally, we show that TP is well suited for practical fine-tuning tasks, such as keyword spotting on the Google Speech Commands dataset, thus paving the way for efficient learning at the edge.

脉冲神经网络边缘学习内存效率局部学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。