系统梳理脉冲神经网络研究进展，聚焦高效大模型未来方向
Toward Large-scale Spiking Neural Networks: A Comprehensive Survey and Future Directions
- 按转换法与代理梯度法分类总结深度脉冲网络学习方法
- 对比现有先进脉冲网络性能，突出脉冲Transformer优势
- 适合关注能效计算与类脑智能的科研人员参考
深度学习已推动人工智能在计算机视觉、语音识别和自然语言处理等领域取得显著进展，尤其大型语言模型的兴起加速了大规模神经网络的研究。然而，算力与能耗的持续增长促使人们寻求更节能的替代方案。受人脑启发，脉冲神经网络（SNN）以事件驱动的脉冲实现低功耗计算。为推动构建高效能大规模脉冲神经网络，本文系统综述现有深度脉冲网络开发方法，重点关注新兴脉冲Transformer。主要贡献包括：(1) 按ANN-to-SNN转换与代理梯度直接训练分类梳理学习方法；(2) 按深度卷积网络（DCNN）与Transformer架构分类总结网络结构；(3) 对当前先进深度SNN进行全面比较，重点分析脉冲Transformer。最后，进一步探讨并提出大规模脉冲神经网络的发展方向。
原文摘要 · Abstract (English)
Deep learning has revolutionized artificial intelligence (AI), achieving remarkable progress in fields such as computer vision, speech recognition, and natural language processing. Moreover, the recent success of large language models (LLMs) has fueled a surge in research on large-scale neural networks. However, the escalating demand for computing resources and energy consumption has prompted the search for energy-efficient alternatives. Inspired by the human brain, spiking neural networks (SNNs) promise energy-efficient computation with event-driven spikes. To provide future directions toward building energy-efficient large SNN models, we present a survey of existing methods for developing deep spiking neural networks, with a focus on emerging Spiking Transformers. Our main contributions are as follows: (1) an overview of learning methods for deep spiking neural networks, categorized by ANN-to-SNN conversion and direct training with surrogate gradients; (2) an overview of network architectures for deep spiking neural networks, categorized by deep convolutional neural networks (DCNNs) and Transformer architecture; and (3) a comprehensive comparison of state-of-the-art deep SNNs with a focus on emerging Spiking Transformers. We then further discuss and outline future directions toward large-scale SNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。