用单次真人演示训练机器人，生成海量仿真数据提升泛化能力
Scaling Single Human Demonstrations for Imitation Learning using Generative Foundational Models
- 从一次真人操作中提取动作信息，生成无限仿真示范
- 成功率平均提升26.6%，训练数据更丰富多样
- 纯仿真训练模型可零样本部署到真实世界
模仿学习是教会机器人新任务的常用方法，但通过遥操作或力控教学收集机器人示范费时费力。相比之下，用人类身体直接演示更简便，数据也更易获取，但迁移到机器人上存在挑战。本文提出Real2Gen，仅需一次真人示范即可训练抓取策略。该方法从示范中提取关键信息并转移至仿真环境，由可编程专家代理任意多次演示任务，生成无限数据用于流匹配策略训练。我们在三个真实任务的人类示范上评估Real2Gen，相比近期基线平均成功率达26.6%提升，且策略泛化能力更强。进一步将纯仿真训练的策略零样本部署于真实世界。数据、代码与训练模型已公开：real2gen.cs.uni-freiburg.de。
原文摘要 · Abstract (English)
Imitation learning is a popular paradigm to teach robots new tasks, but collecting robot demonstrations through teleoperation or kinesthetic teaching is tedious and time-consuming. In contrast, directly demonstrating a task using our human embodiment is much easier and data is available in abundance, yet transfer to the robot can be non-trivial. In this work, we propose Real2Gen to train a manipulation policy from a single human demonstration. Real2Gen extracts required information from the demonstration and transfers it to a simulation environment, where a programmable expert agent can demonstrate the task arbitrarily many times, generating an unlimited amount of data to train a flow matching policy. We evaluate Real2Gen on human demonstrations from three different real-world tasks and compare it to a recent baseline. Real2Gen shows an average increase in the success rate of 26.6% and better generalization of the trained policy due to the abundance and diversity of training data. We further deploy our purely simulation-trained policy zero-shot in the real world. We make the data, code, and trained models publicly available at real2gen.cs.uni-freiburg.de.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。