用行为数据预训练模型,提升脑机接口解码精度与泛化能力。
NeuroPB: Scaling Neural Decoding with Pretrained Behavioral Representations

- 通过大规模行为数据预训练运动编码器,再对齐神经信号。
- 相比从零训练,轨迹解码准确率提升11%(中心向外)和8%(随机目标)。
- 机器人生成数据也可有效迁移,适合数据稀缺场景下的脑机接口应用。
从神经活动解码连续运动轨迹是发展实用脑机接口的关键。然而,现有解码器受限于神经记录的规模与异质性。相比之下,人类、动物、仿真和机器人系统可更易获取大规模行为数据。本文提出NeuroPB框架,通过预训练的行为表征实现神经解码的规模化。该方法先在大规模运动行为数据上预训练运动编码器,再利用少量配对的神经-行为数据将神经活动对齐至行为表征空间。随后优化神经编码器与轻量级运动解码器,以重建连续运动。在多个恒河猴运动数据集上,行为预训练使轨迹解码性能提升,中心向外任务提升11% $R^2$,随机目标任务提升8%。值得注意的是,使用机器人轨迹预训练的表现可媲美使用恒河猴轨迹预训练,表明生物与人工模型共享可迁移的运动学结构。此外,在神经数据固定时,随着机器人预训练数据规模与多样性增加,解码性能持续提升。预训练还增强了跨记录会话、受试者和运动任务的泛化能力,仅需10%校准数据即可达到从零训练的水平。结果表明,行为预训练是一种可扩展的神经解码来源,为有限神经数据下高性能、低校准成本的脑机接口提供了可行路径。
原文摘要 · Abstract (English)
Decoding continuous motor trajectories from neural activity is essential for developing practical brain-computer interfaces (BCIs). However, current neural decoders are constrained by the limited scale and heterogeneity of neural recordings. In contrast, behavioral data can be collected more readily and at substantially larger scale from humans, animals, simulations, and robotic systems. Here, we introduce NeuroPB, a framework that scales neural decoding by transferring knowledge from pretrained behavioral representations. NeuroPB first pretrains a motor encoder on large-scale motor behavior data and then aligns neural activity with the resulting behavioral representation space using a limited set of paired neural-behavioral recordings. A neural encoder and lightweight motor decoder are subsequently optimized to reconstruct continuous movement from the aligned neural representations. Across multiple macaque motor datasets, behavioral pretraining improves trajectory decoding, including an 11% $R^2$ increase on center-out and 8% on random-target compared with training the motor encoder from scratch. Notably, pretraining on robotic trajectories achieves performance comparable to pretraining on macaque trajectories, demonstrating that transferable kinematic structure is shared across biological and artificial models. Moreover, decoding performance improves as the scale and diversity of robotic pretraining data increase, when the amount of neural data is fixed. Pretraining also enhances generalization across recording sessions, subjects, and motor tasks, with only 10% calibration needed to match training from scratch. Overall, these results establish behavioral pretraining as a scalable source for neural decoding and provide a promising route toward high-performance and calibration-efficient BCIs under limited neural data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。