一种高效低功耗的脉冲状态空间模型，适用于可穿戴设备上的超长序列分析
ASecond-Order SpikingSSM for Wearables
- 采用二阶脉冲动力系统与并行扫描算法，实现无矩阵乘法的快速计算
- 在50,000步序列上保持高精度，比传统ANN模型节能52.1倍
- 适合资源受限场景，如可穿戴设备，兼具长程建模与低功耗优势
脉冲神经网络因其能耗低、无需乘法运算和稀疏事件处理而受到关注。同时，状态空间模型通过避免序列长度的二次依赖，成为长序列建模中可扩展的Transformer替代方案。我们提出SHaRe-SSM（Spiking Harmonic Resonate-and-Fire State Space Model），一种用于分类与回归的二阶脉冲状态空间模型，能处理超长序列。该模型在平均性能上超越Transformer和一阶SSM，且完全消除矩阵乘法，非常适合资源受限的应用。为实现数十万时间步的快速计算，我们采用底层动态系统的并行扫描公式。此外，引入基于核函数的脉冲回归器，使序列长达50,000步时仍能准确建模依赖关系。结果表明，SHaRe-SSM在保持卓越长程建模能力的同时，能耗仅为基于ANN的二阶SSM的52.1分之一，是可穿戴等资源受限设备的理想候选方案。
原文摘要 · Abstract (English)
Spiking neural networks have garnered increasing attention due to their energy efficiency, multiplication-free computation, and sparse event-based processing. In parallel, state space models have emerged as scalable alternatives to transformers for long-range sequence modelling by avoiding quadratic dependence on sequence length. We propose SHaRe-SSM (Spiking Harmonic Resonate-and-Fire State Space Model), a second-order spiking SSM for classification and regression on ultra-long sequences. SHaRe-SSM outperforms transformers and first-order SSMs on average while eliminating matrix multiplications, making it highly suitable for resource-constrained applications. To ensure fast computation over tens of thousands of time steps, we leverage a parallel scan formulation of the underlying dynamical system. Furthermore, we introduce a kernel-based spiking regressor, which enables the accurate modelling of dependencies in sequences of up to 50k steps. Our results demonstrate that SHaRe-SSM achieves superior long-range modelling capability with energy efficiency (52.1x less than ANN-based second order SSM), positioning it as a strong candidate for resource-constrained devices such as wearables
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。