用虚拟现实采集人体动作，让机器人学会全身协同操作。
BifrostUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation
- 用轻量VR和特制夹具采集人类关键点轨迹与动作
- 训练高阶策略预测未来动作并映射到机器人执行
- 五种真实场景验证了动作可迁移性，无需实体机器人
高质量示范数据对人形机器人技能学习至关重要，尤其在需要感知、移动与操作协同的全身行为中。现有数据采集多依赖机器人遥操作，受限于硬件可及性、操作者经验与效率。受通用操作接口（UMI）启发，我们提出BifrostUMI，一种便携、无需机器人的全身体操数据采集框架。该框架利用轻量级VR设备与仿UMI夹具，采集稀疏的人体关键点轨迹、手腕视角观测及夹持器动作。这些示范用于训练高层策略以预测未来关键点，再重定向为机器人原生的全身体操参考，并由全身体操控制器执行。五个真实场景实验验证了该框架的有效性，证明所收集示范可用于可迁移的人形机器人全身体操技能学习。
原文摘要 · Abstract (English)
High-quality demonstration data are essential for humanoid robot skill learning, especially for whole-body behaviors that require coordinated perception, locomotion, and manipulation. Existing data-collection methods largely rely on robot teleoperation, which is constrained by hardware accessibility, operator expertise, and limited efficiency. Inspired by the Universal Manipulation Interface (UMI), we propose BifrostUMI, a portable and robot-free framework for humanoid whole-body data collection. BifrostUMI uses lightweight VR devices and UMI-inspired grippers to collect sparse human keypoint trajectories, wrist-view observations, and gripper actions. These demonstrations train a high-level policy to predict future keypoints, which are retargeted to robot-native whole-body references and executed by a whole-body controller. Experiments in five real-world scenarios demonstrate the effectiveness of the proposed framework and validate the collected demonstrations for transferable humanoid whole-body skill learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。