重新审视脉冲网络的重置机制,提出更适合序列建模的新架构。
Revisiting Reset Mechanisms in Spiking Neural Networks for Sequential Modeling: Specialized Discretization for Binary Activated RNN
- 将脉冲网络视为二值化RNN,分析重置与适应期机制
- 提出固定适应期新架构,支持长序列建模与并行训练
- 适合做序列任务的脉冲神经网络研究者参考
在图像识别领域,脉冲神经网络(SNN)性能已接近传统人工神经网络(ANN)。但在序列建模任务中,现有SNN架构面临三大挑战:(1)缺乏有效长程记忆机制;(2)生物启发组件(如重置机制、适应期)在序列任务中的理论研究不足;(3)基于RNN范式的计算方式阻碍了跨时间步的并行训练。本文系统分析了二值激活SNN序列模型中重置操作与适应期的基本机制,重新评估这些生物机制对生成稀疏脉冲模式的必要性,提供新的理论解释,并提出固定适应期脉冲神经网络架构,以支持高效序列建模。
原文摘要 · Abstract (English)
In the field of image recognition, spiking neural networks (SNNs) have achieved performance comparable to conventional artificial neural networks (ANNs). In such applications, SNNs essentially function as traditional neural networks with quantized activation values. This article focuses on an another alternative perspective,viewing SNNs as binary-activated recurrent neural networks (RNNs) for sequential modeling tasks. From this viewpoint, current SNN architectures face several fundamental challenges in sequence modeling: (1) Traditional models lack effective memory mechanisms for long-range sequence modeling; (2) The biological-inspired components in SNNs (such as reset mechanisms and refractory period applications) remain theoretically under-explored for sequence tasks; (3) The RNN-like computational paradigm in SNNs prevents parallel training across different timesteps. To address these challenges, this study conducts a systematic analysis of the fundamental mechanisms underlying reset operations and refractory periods in binary-activated RNN-based SNN sequence models. We re-examine whether such biological mechanisms are strictly necessary for generating sparse spiking patterns, provide new theoretical explanations and insights, and ultimately propose the fixed-refractory-period SNN architecture for sequence modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。