通过集成学习视角提升脉冲神经网络稳定性与性能。
Rethinking Spiking Neural Networks from an Ensemble Learning Perspective
- 将脉冲网络视为共享结构的时序子网络集成。
- 仅用4个时间步达83.20%准确率，显著改善梯度消失问题。
- 无需改动网络结构，适用于语音、图像、点云多任务。
脉冲神经网络（SNNs）具有优异能效但性能受限。本文从集成学习视角出发，将SNN视为共享架构与权重的时序子网络集合，指出初始膜电位差异过大导致子网络输出不稳定，影响整体性能。为此，提出膜电位平滑与相邻子网络引导机制，提升初始电位分布与输出一致性，增强稳定性。该方法促进信息前向传播与梯度反向传播，缓解长期依赖下的梯度消失问题。仅需对脉冲神经元进行微小修改，不改变网络结构，在1D语音、2D物体和3D点云识别任务中均取得一致性能提升。尤其在挑战性CIFAR10-DVS数据集上，仅用4个时间步即达到83.20%准确率，为释放SNN潜力提供新思路。
原文摘要 · Abstract (English)
Spiking neural networks (SNNs) exhibit superior energy efficiency but suffer from limited performance. In this paper, we consider SNNs as ensembles of temporal subnetworks that share architectures and weights, and highlight a crucial issue that affects their performance: excessive differences in initial states (neuronal membrane potentials) across timesteps lead to unstable subnetwork outputs, resulting in degraded performance. To mitigate this, we promote the consistency of the initial membrane potential distribution and output through membrane potential smoothing and temporally adjacent subnetwork guidance, respectively, to improve overall stability and performance. Moreover, membrane potential smoothing facilitates forward propagation of information and backward propagation of gradients, mitigating the notorious temporal gradient vanishing problem. Our method requires only minimal modification of the spiking neurons without adapting the network structure, making our method generalizable and showing consistent performance gains in 1D speech, 2D object, and 3D point cloud recognition tasks. In particular, on the challenging CIFAR10-DVS dataset, we achieved 83.20\% accuracy with only four timesteps. This provides valuable insights into unleashing the potential of SNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。