多级脉冲神经网络提升稀疏性与能效,实现单步推理
All in one timestep: Enhancing Sparsity and Energy efficiency in Multi-level Spiking Neural Networks
- 提出多级脉冲神经元模型,兼顾低量化误差与低延迟
- 图像分类中能效降低2-3倍,脉冲数据推理延迟压缩10倍
- 设计稀疏残差架构,减少20%以上网络活动,适合硬件部署
脉冲神经网络(SNNs)因其事件驱动通信机制,可在专用类脑硬件上实现稀疏且理论低功耗运算。但瞬时脉冲的二值特性导致信息损失,影响精度。为此,本文提出一种多级脉冲神经元模型,在逼近全精度人工神经网络性能的同时,实现低量化误差与最小推理延迟。在主流网络结构与数据集上的实验表明,该模型具有更优的信息压缩能力,可在不损失性能前提下降低延迟。与二值SNN相比,在图像分类任务中能量消耗降低2至3倍,具体取决于量化区间数。在类脑数据上,可将推理延迟压缩至1个时间步,相较先前结果压缩10倍。此外,提出新型稀疏残差架构Sparse-ResNet,通过分析残差连接中的脉冲传播,揭示了脉冲雪崩效应。采用该架构后,图像分类准确率达当前最优,同时网络活跃度较前代脉冲残差网络降低20%以上。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) are one of the most promising bio-inspired neural networks models and have drawn increasing attention in recent years. The event-driven communication mechanism of SNNs allows for sparse and theoretically low-power operations on dedicated neuromorphic hardware. However, the binary nature of instantaneous spikes also leads to considerable information loss in SNNs, resulting in accuracy degradation. To address this issue, we propose a multi-level spiking neuron model able to provide both low-quantization error and minimal inference latency while approaching the performance of full precision Artificial Neural Networks (ANNs). Experimental results with popular network architectures and datasets, show that multi-level spiking neurons provide better information compression, allowing therefore a reduction in latency without performance loss. When compared to binary SNNs on image classification scenarios, multi-level SNNs indeed allow reducing by 2 to 3 times the energy consumption depending on the number of quantization intervals. On neuromorphic data, our approach allows us to drastically reduce the inference latency to 1 timestep, which corresponds to a compression factor of 10 compared to previously published results. At the architectural level, we propose a new residual architecture that we call Sparse-ResNet. Through a careful analysis of the spikes propagation in residual connections we highlight a spike avalanche effect, that affects most spiking residual architectures. Using our Sparse-ResNet architecture, we can provide state-of-the-art accuracy results in image classification while reducing by more than 20% the network activity compared to the previous spiking ResNets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。