arXiv:2411.16061cs.CV2024-11TPAMI被引 105

提出新型脉冲神经网络训练方法,实现低功耗下媲美ANN的图像识别性能。

Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training

论文配图:Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training
图 1 · 摘自论文原文
  • 用整数训练+脉冲驱动推理,优化脉冲发放模式提升效率
  • 在ImageNet上达86.2%准确率,1000万参数模型比现有SNN高7.2%
  • 适合追求低功耗、可扩展性的人工智能部署场景

脑启发脉冲神经网络(SNNs)旨在成为传统人工神经网络(ANNs)的低功耗替代方案。本文针对SNNs与ANNs之间的性能差距以及高训练成本两大挑战,揭示了由二值脉冲机制引发的固有缺陷,并提出一种基于整数训练与脉冲驱动推理的脉冲发放近似(SFA)方法。该方法优化了脉冲神经元的发放模式,实现高效训练、降低功耗、提升性能、便于扩展,并更好利用类脑芯片。同时构建了高效的脉冲驱动Transformer架构和脉冲掩码自编码器,防止SNN在规模扩大时性能下降。在ImageNet-1k上,参数量分别为1000万、1900万、8300万和1.73亿的模型分别取得78.5%、79.8%、84.0%和86.2%的顶级准确率。例如,1000万参数模型在ImageNet上的表现优于现有最佳SNN 7.2%,训练时间提速4.5倍,推理能耗降低3.9倍。该方法在目标检测、语义分割及类脑视觉任务中均验证了有效性与高效性。本工作使SNN在保持低功耗优势的同时达到ANN性能水平,标志着其向通用视觉主干迈进的重要一步。代码已公开于https://github.com/BICLab/Spike-Driven-Transformer-V3。

原文摘要 · Abstract (English)

The ambition of brain-inspired Spiking Neural Networks (SNNs) is to become a low-power alternative to traditional Artificial Neural Networks (ANNs). This work addresses two major challenges in realizing this vision: the performance gap between SNNs and ANNs, and the high training costs of SNNs. We identify intrinsic flaws in spiking neurons caused by binary firing mechanisms and propose a Spike Firing Approximation (SFA) method using integer training and spike-driven inference. This optimizes the spike firing pattern of spiking neurons, enhancing efficient training, reducing power consumption, improving performance, enabling easier scaling, and better utilizing neuromorphic chips. We also develop an efficient spike-driven Transformer architecture and a spike-masked autoencoder to prevent performance degradation during SNN scaling. On ImageNet-1k, we achieve state-of-the-art top-1 accuracy of 78.5\%, 79.8\%, 84.0\%, and 86.2\% with models containing 10M, 19M, 83M, and 173M parameters, respectively. For instance, the 10M model outperforms the best existing SNN by 7.2\% on ImageNet, with training time acceleration and inference energy efficiency improved by 4.5$\times$ and 3.9$\times$, respectively. We validate the effectiveness and efficiency of the proposed method across various tasks, including object detection, semantic segmentation, and neuromorphic vision tasks. This work enables SNNs to match ANN performance while maintaining the low-power advantage, marking a significant step towards SNNs as a general visual backbone. Code is available at https://github.com/BICLab/Spike-Driven-Transformer-V3.

脉冲神经网络低功耗计算Transformer类脑计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。