用少量人类示范自动生成大量机器人操作数据,提升学习效率与泛化能力。
SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deployment

- 从少量人类示范中自动分割、适配并组合技能动作生成新演示
- 在18种任务变体上生成24,000+演示,平均成功率提升24%
- 支持零样本仿真到现实迁移,适用于复杂装配等长时序任务
从人类示范中进行模仿学习是机器人操作的有效方法,但获取大规模数据集成本高,尤其对长时序任务。为此,我们提出SkillMimicGen(SkillGen)——一个从少量人类示范自动生成演示数据集的自动化系统。SkillGen将人类示范分割为操作技能,适应新场景,并通过自由空间移动和转移动作拼接技能。我们还提出混合技能策略(HSP)框架,从SkillGen数据集中学习技能启动、控制和终止组件,使技能可在测试时通过运动规划编排。实验表明,SkillGen显著优于现有数据生成框架,在包含杂乱环境的大场景变化下仍能高效生成数据,使代理平均成功率提高24%。我们在仿真中仅用60个原始示范生成了超过24,000条演示,训练出性能优异、常达近乎完美的HSP代理。最后,我们将SkillGen应用于3个真实机器人操作任务,并在长时序装配任务中实现零样本仿真到现实迁移。更多内容见https://skillgen.github.io。
原文摘要 · Abstract (English)
Imitation learning from human demonstrations is an effective paradigm for robot manipulation, but acquiring large datasets is costly and resource-intensive, especially for long-horizon tasks. To address this issue, we propose SkillMimicGen (SkillGen), an automated system for generating demonstration datasets from a few human demos. SkillGen segments human demos into manipulation skills, adapts these skills to new contexts, and stitches them together through free-space transit and transfer motion. We also propose a Hybrid Skill Policy (HSP) framework for learning skill initiation, control, and termination components from SkillGen datasets, enabling skills to be sequenced using motion planning at test-time. We demonstrate that SkillGen greatly improves data generation and policy learning performance over a state-of-the-art data generation framework, resulting in the capability to produce data for large scene variations, including clutter, and agents that are on average 24% more successful. We demonstrate the efficacy of SkillGen by generating over 24K demonstrations across 18 task variants in simulation from just 60 human demonstrations, and training proficient, often near-perfect, HSP agents. Finally, we apply SkillGen to 3 real-world manipulation tasks and also demonstrate zero-shot sim-to-real transfer on a long-horizon assembly task. Videos, and more at https://skillgen.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。