用回溯蒸馏让脑电波信号实时解码更准更快
REALM: Retrospective Encoder Alignment for LFP Modeling

- 用双向预训练模型教单向学生模型,实现因果实时解码
- 相比现有方法,准确率更高,参数减半、训练时间缩短90%
- 适合无线植入式脑机接口,不依赖高耗能的尖峰信号
尖峰活动是行为解码的主要神经信号,因其时空分辨率高而被广泛使用。然而,随着脑机接口向高通道数和无线化发展,尖峰信号的高采样频率带来高功耗和大带宽需求,成为瓶颈。局部场电位(LFP)代表不同时空尺度的脑活动,具有长期稳定性好、能耗低、带宽要求小等优势。但现有基于LFP的解码模型准确率较低,且多采用非因果架构,难以实现实时部署。为此,本文提出REALM:一种回溯蒸馏框架,实现因果LFP解码。受语音识别中离线到在线蒸馏策略启发,REALM将预训练的多会话双向LFP模型(教师模型)的知识迁移到因果学生模型中。首先,使用掩码自编码目标预训练一个双向Mamba-2教师模型;随后,通过表示对齐与任务监督相结合的目标,将教师模型知识蒸馏到紧凑的学生模型。REALM在行为解码任务中持续优于现有因果与非因果的SOTA方法。特别地,其性能提升的同时,参数量减少2倍,训练时间降低10倍。结果表明,回溯蒸馏有效弥合了离线与实时神经解码之间的差距。REALM证明,仅用LFP信号即可实现媲美尖峰信号的解码性能,为下一代无线植入式脑机接口提供了实用且可扩展的替代方案。
原文摘要 · Abstract (English)
Spike activity has been the dominant neural signal for behavior decoding due to its high spatial and temporal resolution. However, as brain-computer interfaces (BCIs) move toward high channel counts and wireless operation, the high sampling frequency of spike signals becomes a bottleneck due to high power and bandwidth requirements. Local field potentials (LFPs) represent a different spatial-temporal scale of brain activity compared to spikes, offering key advantages including improved long-term stability, reduced energy consumption, and lower bandwidth requirement. Despite these benefits, LFP-based decoding models typically show reduced accuracy and often rely on non-causal architectures that are unsuitable for real-time deployment. To address these challenges, we propose REALM: a retrospective distillation framework that enables causal LFP decoding. Inspired by offline-to-online distillation strategies in speech recognition, REALM transfers representational knowledge from a pretrained multi-session bidirectional LFP model to a causal version for real-time deployment. We first pretrain a bidirectional Mamba-2 teacher model using a masked autoencoding objective. We then distill this teacher model into a compact student model via a combined objective of representation alignment and task supervision. REALM consistently outperforms both causal and non-causal LFP-based SOTA methods for behavior decoding. Notably, our REALM improves decoding performance while achieving a $2\times$ reduction in parameter count and a $10\times$ reduction in training time. These results demonstrate that retrospective distillation effectively bridges the gap between offline and real-time neural decoding. REALM shows that LFP-only models can achieve competitive decoding performance without reliance on spike signals, offering a practical and scalable alternative for next-generation wireless implantable BCIs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。