用仿真数据训练人形机器人抓取行走，实测成功率73.3%。
FetchMan: Learning Visual Humanoid Loco-Manipulation Policies from Simulated Experiences
- 先克隆仿真动作，再用强化学习优化，提升性能上限。
- 在超15万场景上训练，单目标抓取成功率达73.3%。
- 首次实现大规模多物体通用抓取策略，适合机器人研发者参考。
视觉驱动的通用人形机器人运动-操作策略一直是机器人研究的目标。然而，当前依赖大量数据的方法在桌面操作中已难以为继，对需同时行走与平衡的人形机器人更是挑战。我们借鉴运动控制中常用的仿真到现实（sim-to-real）范式，将此方法应用于运动-操作任务。实验发现，仅通过合成演示克隆训练，性能始终受限，无论数据量多大。而采用强化学习突破该瓶颈，使用单稀疏奖励对克隆策略进行流式优化（Flow-GRPO），可显著超越纯模仿学习。上述两阶段形成端到端的仿真到现实管道，覆盖超过15万种场景，用于训练名为FetchMan的模型。我们在新发布的FetchMan-Bench仿真基准上评估，并零样本部署于真实Unitree G1机器人，在未见场景中完成跨场景移动与目标抓取，成功率73.3%。最后，我们将该方法扩展至多物体训练，迈出构建大规模通用运动-操作策略的第一步。
原文摘要 · Abstract (English)
Visual loco-manipulation policies that can generalize to novel scenes and objects have long been a goal of robotics research. However, today's data-hungry algorithms make collecting sufficient demonstrations a struggle for tabletop manipulation, and even more so for humanoids that must also walk and balance. Learning from simulated data and transferring that behavior to the real world, as is commonly done in locomotion, sidesteps this struggle, so we replicate that recipe for loco-manipulation. In doing so, we find that cloning synthetic demonstrations results in a low performance ceiling no matter the amount of training data. Reinforcement learning breaks through it, and refining the cloned policy with Flow-GRPO on a single sparse reward yields performance that synthetic behavior cloning cannot match. Together, these stages form our end-to-end sim-to-real pipeline spanning more than 150,000 scenes, which we use to train FetchMan. We evaluate it on FetchMan-Bench, a simulation benchmark we release, and deploy it zero-shot on a real Unitree G1, where our single-object reach-and-pick policy walks to and grasps a target across unseen scenes at 73.3% success. Finally, we extend this recipe to multi-object training, a first step toward loco-manipulation generalist policies at this data scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。