将SSM模型部署到神经形态芯片Loihi 2，实现超低功耗实时序列处理。
A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processing
- 将S4D模型映射至神经形态芯片Loihi 2，实现逐标记推理。
- 相比Jetson GPU，能耗降低1000倍，延迟降低75倍，吞吐量提升75倍。
- 适合对能效和实时性要求极高的边缘计算场景。
深度状态空间模型（SSM）在长序列建模任务中表现优异。尽管其递归结构可在训练时以卷积或并行扫描形式高效实现，但当前在GPU上难以高效进行逐标记的递归处理。本文首次在英特尔最新神经形态处理器Loihi 2上实现了SSM S4D的高效逐标记推理。我们在sMNIST、psMNIST和sCIFAR数据集上对比了该实现与在Jetson Orin Nano（Jetson）上的递归和卷积式S4D实现。在离线批处理模式下，Jetson表现更优；但在逐标记处理场景中，Loihi 2能耗降低1000倍，延迟降低75倍，吞吐量提升75倍。这为SSM在实时流式应用中的高效部署开辟了新路径。
原文摘要 · Abstract (English)
Deep State-Space Models (SSM) demonstrate state-of-the art performance on long-range sequence modeling tasks. While the recurrent structure of SSMs can be efficiently implemented as a convolution or as a parallel scan during training, recurrent token-by-token processing cannot currently be implemented efficiently on GPUs. Here, we demonstrate efficient token-by-token inference of the SSM S4D on Intel's Loihi 2 state-of-the-art neuromorphic processor. We compare this first ever neuromorphic-hardware implementation of an SSM on sMNIST, psMNIST, and sCIFAR to a recurrent and a convolutional implementation of S4D on Jetson Orin Nano (Jetson). While we find Jetson to perform better in an offline sample-by-sample based batched processing mode, Loihi 2 outperforms during token-by-token based processing, where it consumes 1000 times less energy with a 75 times lower latency and a 75 times higher throughput compared to the recurrent implementation of S4D on Jetson. This opens up new avenues towards efficient real-time streaming applications of SSMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。