用变化编码降低脉冲数，让神经网络转换更省电高效
Differential Coding for Training-Free ANN-to-SNN Conversion
- 通过传递激活变化而非原始值，减少脉冲数量
- 在多个CNN和Transformer模型上提升精度并降低能耗
- 适合需要低功耗部署的AI芯片开发者
脉冲神经网络（SNN）因低功耗潜力巨大而备受关注。将人工神经网络（ANN）转换为SNN是实现高性能SNN的有效途径。然而，现有方法多基于率编码，需大量脉冲和更长时步，导致能耗与延迟增加。本文提出差分编码用于ANN到SNN的转换，通过传输速率变化而非直接传递速率，显著减少脉冲数与能耗，并在各类层中验证其有效性。同时提出阈值迭代法，根据激活分布优化ReLU转脉冲神经元的阈值。在多个卷积神经网络（CNN）与Transformer模型上的实验表明，该方法显著提升准确率并降低能耗，尤其结合阈值迭代法后达到当前最优性能。代码已开源：https://github.com/h-z-h-cell/ANN-to-SNN-DCGS。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) exhibit significant potential due to their low energy consumption. Converting Artificial Neural Networks (ANNs) to SNNs is an efficient way to achieve high-performance SNNs. However, many conversion methods are based on rate coding, which requires numerous spikes and longer time-steps compared to directly trained SNNs, leading to increased energy consumption and latency. This article introduces differential coding for ANN-to-SNN conversion, a novel coding scheme that reduces spike counts and energy consumption by transmitting changes in rate information rather than rates directly, and explores its application across various layers. Additionally, the threshold iteration method is proposed to optimize thresholds based on activation distribution when converting Rectified Linear Units (ReLUs) to spiking neurons. Experimental results on various Convolutional Neural Networks (CNNs) and Transformers demonstrate that the proposed differential coding significantly improves accuracy while reducing energy consumption, particularly when combined with the threshold iteration method, achieving state-of-the-art performance. The source codes of the proposed method are available at https://github.com/h-z-h-cell/ANN-to-SNN-DCGS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。