用自然语言控制人形机器人完成复杂动作,支持真实世界稳定执行。
FRoM-W1: Towards General Humanoid Whole-Body Control with Language Instructions
- 通过语言驱动生成人类动作,再转换为机器人可执行指令
- 在H1/G1机器人上实现95%以上动作成功率,优于现有方法
- 适合研究人形机器人智能、具身交互与多模态控制的开发者
人形机器人能完成打招呼、跳舞甚至后空翻等动作,但这些动作通常需硬编码或针对性训练,限制了灵活性。本文提出FRoM-W1,一个开源框架,实现基于自然语言的通用人形机器人全身运动控制。该框架分两阶段运行:(a) H-GPT:利用大规模人类数据,训练一个大规模语言驱动的人类全身动作生成模型,采用思维链技术提升指令理解泛化能力;(b) H-ACT:将生成的人类动作重定向为机器人特定动作,通过物理仿真中预训练并强化学习微调的运动控制器,使机器人在真实世界中准确稳定执行动作。最终通过模块化仿真到现实迁移部署。我们在Unitree H1和G1机器人上广泛评估,结果表明在HumanML3D-X基准上表现优异,强化学习微调显著提升动作跟踪精度与任务成功率(达95%以上)。我们开源完整框架,以推动人形智能发展。
原文摘要 · Abstract (English)
Humanoid robots are capable of performing various actions such as greeting, dancing and even backflipping. However, these motions are often hard-coded or specifically trained, which limits their versatility. In this work, we present FRoM-W1, an open-source framework designed to achieve general humanoid whole-body motion control using natural language. To universally understand natural language and generate corresponding motions, as well as enable various humanoid robots to stably execute these motions in the physical world under gravity, FRoM-W1 operates in two stages: (a) H-GPT: utilizing massive human data, a large-scale language-driven human whole-body motion generation model is trained to generate diverse natural behaviors. We further leverage the Chain-of-Thought technique to improve the model's generalization in instruction understanding. (b) H-ACT: After retargeting generated human whole-body motions into robot-specific actions, a motion controller that is pretrained and further fine-tuned through reinforcement learning in physical simulation enables humanoid robots to accurately and stably perform corresponding actions. It is then deployed on real robots via a modular simulation-to-reality module. We extensively evaluate FRoM-W1 on Unitree H1 and G1 robots. Results demonstrate superior performance on the HumanML3D-X benchmark for human whole-body motion generation, and our introduced reinforcement learning fine-tuning consistently improves both motion tracking accuracy and task success rates of these humanoid robots. We open-source the entire FRoM-W1 framework and hope it will advance the development of humanoid intelligence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。