arXiv:2505.13741cs.CVcs.NE2025-05

提出fBP算法,让脉冲神经网络训练无需对称权重,大幅降低硬件传输开销。

Frozen Backpropagation: Relaxing Weight Symmetry in Deep Spiking Neural Networks

  • 用周期冻结反馈权重的方式更新前向权重,打破传统对称性限制
  • 在图像识别任务中达到接近反向传播的准确率,传输成本降低至原有1/10000
  • 适合需低功耗、高效率的类脑芯片上在线学习场景

在类脑硬件上直接训练脉冲神经网络(SNN)相比基于GPU的训练可显著降低能耗。然而,在此类硬件上实现反向传播(BP)存在挑战:前向与反向传播通常由具有不同权重的两个网络完成,为计算正确梯度,前向与反馈权重必须保持对称,这需要在两网络间频繁传输权重,带来额外硬件开销和能耗。为此,我们提出冻结反向传播(fBP),一种可在分离网络设置下放松权重对称性的BP训练算法。fBP通过周期性冻结反馈权重来计算梯度并更新前向权重,减少训练过程中的权重传输次数,降低同步开销。为进一步提升传输效率,我们提出了三种不同程度计算复杂度的局部权重传输方案,仅在特定时刻传输部分权重。我们在时序编码和率编码的SNN上评估了该方法,在图像识别任务中表现优于现有对称性缓解方法,达到与标准BP相当的精度,同时显著降低传输成本。结合局部传输策略,成本可进一步降低高达10,000倍,仅伴随适度的精度损失。本工作为支持基于BP的片上学习的类脑硬件设计提供了重要指导。

原文摘要 · Abstract (English)

Direct training of Spiking Neural Networks (SNNs) on neuromorphic hardware can greatly reduce energy costs compared to GPU-based training. However, implementing Backpropagation (BP) on such hardware is challenging because forward and backward passes are typically performed by separate networks with distinct weights. To compute correct gradients, forward and feedback weights must remain symmetric during training, necessitating weight transport between the two networks. This symmetry requirement imposes hardware overhead and increases energy costs. To address this issue, we introduce Frozen Backpropagation (\textsc{fBP}), a BP-based training algorithm relaxing weight symmetry in settings with separate networks. fBP updates forward weights by computing gradients with periodically frozen feedback weights, reducing weight transports during training and minimizing synchronization overhead. To further improve transport efficiency, we propose three partial weight transport schemes of varying computational complexity, where only a subset of weights is transported at a time. We evaluate our methods on image recognition tasks using both temporally and rate-coded SNNs, and compare them to existing approaches addressing the weight symmetry requirement. Our results show that fBP outperforms these methods and achieves accuracy comparable to BP while significantly lowering transport costs. With partial weight transport, fBP can further lower those costs by up to 10,000x at the expense of moderate accuracy loss. This work provides insights for guiding the design of neuromorphic hardware incorporating BP-based on-chip learning.

脉冲神经网络反向传播类脑计算低功耗

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。