混合状态空间模型实现实时神经解码,兼顾速度与泛化能力。
Generalizable, real-time neural decoding with hybrid state-space models
- 结合交叉注意力与递归状态空间模型,实现快速在线预测。
- 在猴运动任务上表现接近顶级Transformer,推理速度提升9倍。
- 跨物种预训练提升人类书写/语音解码性能,适合临床应用。
实时解码神经活动是神经科学与神经技术的核心,如闭环实验和脑机接口,要求模型满足严格延迟约束。传统方法(如简单RNN)虽快且轻量,但泛化能力差;而基于Transformer的方法虽有强泛化性,但计算开销大,不适用于低资源或实时场景。为此,本文提出POSSM:一种混合架构,通过交叉注意力对单个尖峰进行编码,并以递归状态空间模型(SSM)为骨干,实现(1)神经活动的快速因果在线预测,(2)通过多数据集预训练高效泛化至新会话、新个体和新任务。我们在猕猴皮层运动任务中评估其解码性能与推理速度,结果表明其可拓展至临床应用——人类书写与语音解码。值得注意的是,用猕猴运动皮层数据预训练显著提升了人类书写任务的解码性能,展现出跨物种迁移的巨大潜力。在所有任务中,POSSM达到与最先进Transformer相当的精度,但推理成本仅为几分之一(GPU上最快达9倍加速)。这些结果表明,混合状态空间模型是实现高精度、高速度与强泛化性的理想选择。
原文摘要 · Abstract (English)
Real-time decoding of neural activity is central to neuroscience and neurotechnology applications, from closed-loop experiments to brain-computer interfaces, where models are subject to strict latency constraints. Traditional methods, including simple recurrent neural networks, are fast and lightweight but often struggle to generalize to unseen data. In contrast, recent Transformer-based approaches leverage large-scale pretraining for strong generalization performance, but typically have much larger computational requirements and are not always suitable for low-resource or real-time settings. To address these shortcomings, we present POSSM, a novel hybrid architecture that combines individual spike tokenization via a cross-attention module with a recurrent state-space model (SSM) backbone to enable (1) fast and causal online prediction on neural activity and (2) efficient generalization to new sessions, individuals, and tasks through multi-dataset pretraining. We evaluate POSSM's decoding performance and inference speed on intracortical decoding of monkey motor tasks, and show that it extends to clinical applications, namely handwriting and speech decoding in human subjects. Notably, we demonstrate that pretraining on monkey motor-cortical recordings improves decoding performance on the human handwriting task, highlighting the exciting potential for cross-species transfer. In all of these tasks, we find that POSSM achieves decoding accuracy comparable to state-of-the-art Transformers, at a fraction of the inference cost (up to 9x faster on GPU). These results suggest that hybrid SSMs are a promising approach to bridging the gap between accuracy, inference speed, and generalization when training neural decoders for real-time, closed-loop applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。