用脑电+肌电解码手部运动，再通过智能助手优化轨迹，让机械臂更精准抓取。
Copilot-Assisted Second-Thought Framework for Brain-to-Robot Hand Motion Decoding
- 融合卷积与注意力机制，从脑电信号中解码手部运动轨迹。
- 单人测试下相关系数达0.995（X轴），跨人测试仍超0.96，性能出色。
- 引入智能辅助过滤低可信度点，仅丢弃20%数据就提升解码精度。
从脑电图（EEG）解码运动学参数是发展运动相关脑机接口的重要方向。传统方法多依赖卷积神经网络（CNN）或循环神经网络（RNN），而基于Transformer的模型在建模长序列脑电信号方面表现优异。本研究提出一种CNN-注意力混合模型，用于解码抓握-举起任务中的手部运动学，实现在同被试实验中表现强劲。进一步拓展至脑电-肌电（EEG-EMG）多模态解码，显著提升性能。同被试测试中，拇指与食指中点轨迹在X、Y、Z轴上的皮尔逊相关系数（PCC）分别为0.9854、0.9946和0.9065；跨被试测试结果为0.9643、0.9795和0.5852。两种模态解码轨迹用于控制MuJoCo仿真环境中的Franka Panda机械臂。为提升轨迹保真度，引入基于有限状态机的运动状态感知批评者，实现低置信度点过滤。后处理使纯脑电解码的总体同被试PCC提升至0.93，同时剔除数据点少于20%。
原文摘要 · Abstract (English)
Motor kinematics prediction (MKP) from electroencephalography (EEG) is an important research area for developing movement-related brain-computer interfaces (BCIs). While traditional methods often rely on convolutional neural networks (CNNs) or recurrent neural networks (RNNs), Transformer-based models have shown strong ability in modeling long sequential EEG data. In this study, we propose a CNN-attention hybrid model for decoding hand kinematics from EEG during grasp-and-lift tasks, achieving strong performance in within-subject experiments. We further extend this approach to EEG-EMG multimodal decoding, which yields substantially improved results. Within-subject tests achieve PCC values of 0.9854, 0.9946, and 0.9065 for the X, Y, and Z axes, respectively, computed on the midpoint trajectory between the thumb and index finger, while cross-subject tests result in 0.9643, 0.9795, and 0.5852. The decoded trajectories from both modalities are then used to control a Franka Panda robotic arm in a MuJoCo simulation. To enhance trajectory fidelity, we introduce a copilot framework that filters low-confidence decoded points using a motion-state-aware critic within a finite-state machine. This post-processing step improves the overall within-subject PCC of EEG-only decoding to 0.93 while excluding fewer than 20% of the data points.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。