用随机投影方法训练脉冲神经网络,更高效且符合生物原理。
Training Spiking Neural Networks via Augmented Direct Feedback Alignment
- 采用增强型直接反馈对齐法,无需梯度反向传播
- 在不依赖精确系统先验的情况下达到媲美传统方法的性能
- 适合在类脑硬件上实现,兼具可实施性与生物合理性
脉冲神经网络(SNN)受真实神经元机制启发,通过离散动作电位传递信息,其稀疏异步特性使其具备极高的能效,是部署于类脑器件的有前景方案。然而,由于神经元不可导,传统基于误差反向传播(BP)和近似梯度设计的训练方法难以实现且缺乏生物学合理性。本文提出使用增强型直接反馈对齐(aDFA),一种基于随机投影的无梯度训练方法,仅需前向过程的部分信息,实现简便且生物合理。通过遗传算法系统分析其有效工作范围与最优设置,并验证了该方法在不同SNN特性下的稳定性和优越性,性能优于传统BP与常规DFA。所提aDFA-SNN方案无需精确系统先验即可取得竞争性表现,为物理实现脉冲神经网络提供了重要参考。
原文摘要 · Abstract (English)
Spiking neural networks (SNNs), the models inspired by the mechanisms of real neurons in the brain, transmit and represent information by employing discrete action potentials or spikes. The sparse, asynchronous properties of information processing make SNNs highly energy efficient, leading to SNNs being promising solutions for implementing neural networks in neuromorphic devices. However, the nondifferentiable nature of SNN neurons makes it a challenge to train them. The current training methods of SNNs that are based on error backpropagation (BP) and precisely designing surrogate gradient are difficult to implement and biologically implausible, hindering the implementation of SNNs on neuromorphic devices. Thus, it is important to train SNNs with a method that is both physically implementatable and biologically plausible. In this paper, we propose using augmented direct feedback alignment (aDFA), a gradient-free approach based on random projection, to train SNNs. This method requires only partial information of the forward process during training, so it is easy to implement and biologically plausible. We systematically demonstrate the feasibility of the proposed aDFA-SNNs scheme, propose its effective working range, and analyze its well-performing settings by employing genetic algorithm. We also analyze the impact of crucial features of SNNs on the scheme, thus demonstrating its superiority and stability over BP and conventional direct feedback alignment. Our scheme can achieve competitive performance without accurate prior knowledge about the utilized system, thus providing a valuable reference for physically training SNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。