用脑电波控制人形机器人,实现无语言沟通的智能协作。
E2H: A Two-Stage Non-Invasive Neural Signal Driven Humanoid Robotic Whole-Body Control Framework
- 分两阶段:先解码脑电信号为动作关键词,再用大模型生成精准动作
- 在低质量非侵入式信号下仍能实现人形机器人全身控制
- 适合言语障碍者、太空或水下等特殊场景的人机协同
近年来,人形机器人通过分层强化学习控制与大语言模型规划,显著提升了复杂任务执行能力。然而,人类因素的研究仍相对滞后。直接用脑波操控人形机器人虽常见于科幻作品(如《环太平洋》《高达》),但现实挑战巨大。本文提出E2H(EEG-to-Humanoid)框架,首次实现高频率非侵入性神经信号对人形机器人的全身控制。针对非侵入式信号质量低、难以精确解码空间轨迹的问题,E2H采用创新的两阶段设计:第一阶段将脑电图(EEG)信号解码为语义动作关键词;第二阶段借助大语言模型生成运动并结合精确动作模仿控制策略,完成机器人控制。该方法为无法发声场景(如言语障碍、太空探索、深海作业)提供了全新人机交互路径,极大拓展了人机协作边界。
原文摘要 · Abstract (English)
Recent advancements in humanoid robotics, including the integration of hierarchical reinforcement learning-based control and the utilization of LLM planning, have significantly enhanced the ability of robots to perform complex tasks. In contrast to the highly developed humanoid robots, the human factors involved remain relatively unexplored. Directly controlling humanoid robots with the brain has already appeared in many science fiction novels, such as Pacific Rim and Gundam. In this work, we present E2H (EEG-to-Humanoid), an innovative framework that pioneers the control of humanoid robots using high-frequency non-invasive neural signals. As the none-invasive signal quality remains low in decoding precise spatial trajectory, we decompose the E2H framework in an innovative two-stage formation: 1) decoding neural signals (EEG) into semantic motion keywords, 2) utilizing LLM facilitated motion generation with a precise motion imitation control policy to realize humanoid robotics control. The method of directly driving robots with brainwave commands offers a novel approach to human-machine collaboration, especially in situations where verbal commands are impractical, such as in cases of speech impairments, space exploration, or underwater exploration, unlocking significant potential. E2H offers an exciting glimpse into the future, holding immense potential for human-computer interaction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。