arXiv:2506.14464cs.NEcs.AI2025-06被引 3

提出HYPR方法,让脉冲神经网络实现高效在线训练。

A Scalable Hybrid Training Approach for Recurrent Spiking Neural Networks

  • 结合前向梯度与并行计算,实现近似在线学习
  • 内存消耗恒定,不受序列长度影响,速度更快
  • 适合在类脑硬件上训练复杂脉冲网络,尤其适配振荡型神经元

循环脉冲神经网络(RSNN)可在类脑系统中高效实现,但其基于梯度的训练通常依赖标准数字硬件上的时间反向传播(BPTT),存在无法在线训练、内存随计算步数线性增长的问题。相比之下,前向传播梯度方法支持在线学习且内存与序列长度无关,但执行速度慢、性能较差。本文提出HYPR(HYbrid PRopagation)方法,融合并行化与近似前向学习,在保持恒定内存开销的同时实现高吞吐量在线训练。该方法可对包含任意非线性脉冲神经元模型的子序列并行更新参数。实验表明,具有亚阈值振荡动力学的神经元模型特别适合用HYPR训练,其任务性能与BPTT相比差距极小,达到前所未有的低水平。

原文摘要 · Abstract (English)

Recurrent spiking neural networks (RSNNs) can be implemented very efficiently in neuromorphic systems. Nevertheless, training of these models with powerful gradient-based learning algorithms is mostly performed on standard digital hardware using Backpropagation through time (BPTT). However, BPTT has substantial limitations. It does not permit online training and its memory consumption scales linearly with the number of computation steps. In contrast, learning methods using forward propagation of gradients operate in an online manner with a memory consumption independent of the number of time steps. These methods enable SNNs to learn from continuous, infinite-length input sequences. Yet, slow execution speed on conventional hardware as well as inferior performance has hindered their widespread application. In this work, we introduce HYbrid PRopagation (HYPR) that combines the efficiency of parallelization with approximate online forward learning. Our algorithm yields high-throughput online learning through parallelization, paired with constant, i.e., sequence length independent, memory demands. HYPR enables parallelization of parameter update computation over the sub sequences for RSNNs consisting of almost arbitrary non-linear spiking neuron models. We apply HYPR to networks of spiking neurons with oscillatory subthreshold dynamics. We find that this type of neuron model is particularly well trainable by HYPR, resulting in an unprecedentedly low task performance gap between approximate forward gradient learning and BPTT.

脉冲神经网络在线学习类脑计算并行训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。