用虚拟现实采集人体动作数据，让机器人学会全身协同操作。
HumanoidUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation
- 通过轻量VR设备和仿UMI夹具收集人体关键点与操作数据。
- 训练出可预测未来动作的关键点模型，成功迁移到真实机器人上。
- 无需机器人参与即可生成高质量演示数据，适合复杂操作学习。
高质量示范数据对人形机器人技能学习至关重要，尤其在需要感知、移动与操作协同的全身行为中。现有数据采集方法主要依赖机器人遥操作，受限于硬件可及性、操作员经验与效率。受通用操作接口（UMI）启发，我们提出HumanoidUMI——一种便携、无需机器人的全身体操数据采集框架。该框架利用轻量级VR设备与仿UMI夹具，收集稀疏的人体关键点轨迹、手腕视角观测与夹持器动作。这些示范数据用于训练高层策略以预测未来关键点，再将其重定向为机器人原生的全身体控参考，并由全身体控器执行。在五个真实场景中的实验验证了该框架的有效性，证明所采集数据可用于可迁移的人形机器人全身体操技能学习。
原文摘要 · Abstract (English)
High-quality demonstration data are essential for humanoid robot skill learning, especially for whole-body behaviors that require coordinated perception, locomotion, and manipulation. Existing data-collection methods largely rely on robot teleoperation, which is constrained by hardware accessibility, operator expertise, and limited efficiency. Inspired by the Universal Manipulation Interface (UMI), we propose HumanoidUMI, a portable and robot-free framework for humanoid whole-body data collection. HumanoidUMI uses lightweight VR devices and UMI-inspired grippers to collect sparse human keypoint trajectories, wrist-view observations, and gripper actions. These demonstrations train a high-level policy to predict future keypoints, which are retargeted to robot-native whole-body references and executed by a whole-body controller. Experiments in five real-world scenarios demonstrate the effectiveness of the proposed framework and validate the collected demonstrations for transferable humanoid whole-body skill learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。