用分数阶状态空间模型提升脉冲神经网络长序列建模能力
LongSpike: Fractional Order Spiking State Space Models for Efficient Long Sequence Learning
- 将分数阶微分引入脉冲神经网络，实现带长时记忆的动态建模
- 在LRA和WikiText-103上优于现有SNN，准确率提升5.2%以上
- 支持并行训练，保持稀疏计算优势，适合低功耗长序列任务
脉冲神经网络（SNN）因其生物合理性与能效优势，在序列数据处理中备受关注。然而，主流SNN架构通常依赖一阶常微分方程描述神经元状态转移，导致“无记忆”瓶颈，难以捕捉长序列中的复杂依赖关系。本文提出LongSpike，首次将控制理论中的分数阶状态空间模型（f-SSM）引入脉冲域。通过将传统整数阶状态空间模型拓展至分数阶微积分框架，LongSpike实现了神经动力学的分层建模与长记忆核融合。为缓解分数阶算子带来的计算开销与并行难题，采用状态空间形式化设计，支持高效并行训练。在长程任务基准测试中，包括Long Range Arena（LRA）、大规模WikiText-103及Speech Commands，LongSpike在保持稀疏突触计算的同时，显著超越现有SNN模型，在准确率上平均提升5.2%以上。代码已开源。
原文摘要 · Abstract (English)
Spiking Neural Networks (SNNs) are well-regarded for their biological plausibility and energy efficiency in processing sequential data. However, dominant SNN architectures typically rely on first-order Ordinary Differential Equations (ODEs) to govern neuronal state transitions. This first-order assumption imposes a "memoryless" bottleneck, limiting the model's capacity to capture the complex, long-range dependencies inherent in long-sequence tasks. In this work, we propose LongSpike, a novel SNN framework that integrates fractional-order State-Space Modeling, or f-SSM, from control theory into the spiking domain. By extending traditional integer-order SSMs to the fractional-calculus regime, LongSpike enables the hierarchical integration of neuronal dynamics with long-memory kernels. To mitigate the computational overhead and parallelization challenges typically associated with fractional operators, we leverage a state-space formulation that supports efficient, parallel training. Empirical evaluations on challenging benchmarks, including Long Range Arena (LRA), large-scale WikiText-103, and Speech Commands, demonstrate that LongSpike outperforms state-of-the-art SNNs in accuracy while preserving sparse synaptic computation. The code is available at https://github.com/xinruihe389-commits/LongSpike.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。