用低秩进化策略训练脉冲神经网络，省时省力且无需反向传播。
Gradient-Free Training of Spiking Neural Networks via Low-Rank Evolution Strategies
- 通过低秩分解减少进化策略的参数扰动内存开销。
- 在N-MNIST上达79.21%准确率，单代耗时降低2.23倍。
- 适合在类脑硬件上实现无梯度训练，兼顾效率与可部署性。
脉冲神经网络（SNN）在类脑硬件上具有出色的能效优势，但其训练困难，因尖峰阈值为离散不可导。替代方法如代理梯度虽绕过此问题，却依赖反向传播，不适用于片上学习。进化策略（ES）是无梯度的自然选择，但计算成本随参数量增长，难以用于大权重矩阵。本文提出EGGROLL，一种对ES扰动进行低秩分解的方法，将每代内存需求从𝒪(mn)降至𝒪(r(m+n))。结合泄漏积分-放电（LIF）SNN在N-MNIST上的实验表明，该方法实现79.21%测试准确率，同时相比全秩ES将单代运行时间缩短2.23倍。结果证明EGGROLL可用于SNN训练，具备清晰的精度-速度权衡，且无需代理梯度，兼容类脑硬件训练。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) offer compelling energy efficiency on neuromorphic hardware, yet their training remains challenging because the discrete spike threshold is non-differentiable. Surrogate-gradient methods sidestep this by approximating the derivative, but they impose backpropagation infrastructure that is incompatible with on-chip learning. Evolution Strategies (\es) are a natural gradient-free alternative, yet their computational cost scales with the number of parameters, making them impractical for large weight matrices. We present a method for training SNNs using EGGROLL, a low-rank factorisation of ES perturbations that reduces per-generation memory from $\mathcal{O}(mn)$ to $\mathcal{O}(r(m{+}n))$. Combining EGGROLL with a Leaky Integrate-and-Fire SNN on N-MNIST, we demonstrate that gradient-free training achieves 79.21% test accuracy while reducing per-generation wall-clock time by 2.23$\times$ relative to full-rank ES. Our results demonstrate EGGROLL is viable for SNN training, with a clear accuracy-speed tradeoff, compatible with training on neuromorphic hardware without surrogate gradients.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。