用随机特征法训练脉冲神经网络，无需近似梯度，效果更好。
Random Feature Spiking Neural Networks
- 借鉴随机特征方法，直接训练脉冲神经网络，不依赖梯度近似。
- S-SWIM算法在时间序列预测上达到高精度，且可作为训练起点。
- 相比随机初始化权重，性能更优，兼具速度与可解释性。
脉冲神经网络（SNNs）作为机器学习模型近年来受到关注，因其可能比传统人工神经网络更节能。但其非可微性和稀疏性使基于梯度反向传播的训练变得困难。本文将人工神经网络中的随机特征方法（RFMs）引入到脉冲响应模型（SRM）SNNs中，提出一种新算法S-SWIM，实现无需近似脉冲函数梯度的端到端训练。该方法数据驱动、快速高效、可解释性强。理论分析与数值实验表明，S-SWIM在时间序列预测任务中表现优异，可作为独立训练策略，也可用于梯度训练前的初始化。消融实验显示其优于随机权重采样。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) as Machine Learning (ML) models have recently received a lot of attention as a potentially more energy-efficient alternative to conventional Artificial Neural Networks. The non-differentiability and sparsity of the spiking mechanism can make these models very difficult to train with algorithms based on propagating gradients through the spiking non-linearity. We address this problem by adapting the paradigm of Random Feature Methods (RFMs) from Artificial Neural Networks (ANNs) to Spike Response Model (SRM) SNNs. This approach allows training of SNNs without approximation of the spike function gradient. Concretely, we propose a novel data-driven, fast, high-performance, and interpretable algorithm for end-to-end training of SNNs inspired by the SWIM algorithm for RFM-ANNs, which we coin S-SWIM. We provide a thorough theoretical discussion and supplementary numerical experiments showing that S-SWIM can reach high accuracies on time series forecasting as a standalone strategy and serve as an effective initialisation strategy before gradient-based training. Additional ablation studies show that our proposed method performs better than random sampling of network weights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。