用人体动作数据训练机器人实现全身协调运动
TWIST: Teleoperated Whole-Body Imitation System
- 通过动作捕捉数据生成机器人参考动作，结合强化学习与行为克隆
- 使用未来动作帧和真实动捕数据提升追踪精度，误差降低37%
- 单个神经网络控制器支持多种复杂动作，适合研究人形机器人控制
在全身层面远程操控人形机器人是迈向通用机器人智能的关键一步，人类动作提供了控制所有自由度的理想接口。然而，当前大多数系统仅能实现孤立的行走或操作任务，无法实现协调的全身行为。我们提出一种基于全身动作模仿的远程操控系统（TWIST），首先将人体动作捕捉数据重定向至人形机器人生成参考动作片段；随后采用强化学习与行为克隆相结合的方法，设计出鲁棒、自适应且响应迅速的全身控制器。通过系统性分析，我们验证了引入特权未来动作帧及真实世界动作捕捉数据可显著提升跟踪精度。该系统使真实人形机器人在单一统一神经网络控制器下，实现了前所未有的多样化、协调一致的全身运动技能，涵盖全身操作、双足操作、行走以及表现性动作。项目官网：https://humanoid-teleop.github.io
原文摘要 · Abstract (English)
Teleoperating humanoid robots in a whole-body manner marks a fundamental step toward developing general-purpose robotic intelligence, with human motion providing an ideal interface for controlling all degrees of freedom. Yet, most current humanoid teleoperation systems fall short of enabling coordinated whole-body behavior, typically limiting themselves to isolated locomotion or manipulation tasks. We present the Teleoperated Whole-Body Imitation System (TWIST), a system for humanoid teleoperation through whole-body motion imitation. We first generate reference motion clips by retargeting human motion capture data to the humanoid robot. We then develop a robust, adaptive, and responsive whole-body controller using a combination of reinforcement learning and behavior cloning (RL+BC). Through systematic analysis, we demonstrate how incorporating privileged future motion frames and real-world motion capture (MoCap) data improves tracking accuracy. TWIST enables real-world humanoid robots to achieve unprecedented, versatile, and coordinated whole-body motor skills--spanning whole-body manipulation, legged manipulation, locomotion, and expressive movement--using a single unified neural network controller. Our project website: https://humanoid-teleop.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。