用速率编码简化脉冲神经网络训练，显著降低计算开销。
Advancing Training Efficiency of Deep Spiking Neural Networks through Rate-based Backpropagation
- 基于脉冲频率设计新反向传播，避免复杂时间导数计算。
- 在CIFAR和ImageNet上性能接近传统方法，内存占用更低。
- 适合资源受限场景下的高效脉冲神经网络训练。
近期研究发现，基于代理梯度的时空反向传播（BPTT）在训练深度脉冲神经网络（SNNs）时主要依赖速率编码进行信息表征。受此启发，我们提出速率编码反向传播，一种专为利用速率编码特性而设计的训练策略，通过聚焦平均动态行为减少对精确时间导数的依赖，从而简化计算图，降低训练过程中的内存与计算开销。我们通过理论分析与实验观察验证了该方法与BPTT间梯度近似的合理性。在CIFAR-10、CIFAR-100、ImageNet及CIFAR10-DVS上的大量实验表明，本方法性能可媲美BPTT基准，且优于现有高效训练技术。借助速率编码的内在优势，本工作为资源受限环境下的可扩展、高效率SNN训练奠定了基础。代码已公开于https://github.com/Tab-ct/rate-based-backpropagation。
原文摘要 · Abstract (English)
Recent insights have revealed that rate-coding is a primary form of information representation captured by surrogate-gradient-based Backpropagation Through Time (BPTT) in training deep Spiking Neural Networks (SNNs). Motivated by these findings, we propose rate-based backpropagation, a training strategy specifically designed to exploit rate-based representations to reduce the complexity of BPTT. Our method minimizes reliance on detailed temporal derivatives by focusing on averaged dynamics, streamlining the computational graph to reduce memory and computational demands of SNNs training. We substantiate the rationality of the gradient approximation between BPTT and the proposed method through both theoretical analysis and empirical observations. Comprehensive experiments on CIFAR-10, CIFAR-100, ImageNet, and CIFAR10-DVS validate that our method achieves comparable performance to BPTT counterparts, and surpasses state-of-the-art efficient training techniques. By leveraging the inherent benefits of rate-coding, this work sets the stage for more scalable and efficient SNNs training within resource-constrained environments. Our code is available at https://github.com/Tab-ct/rate-based-backpropagation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。