用60个真人示范生成2.1万条机器人双臂操作数据，解决抓取训练数据难问题。
DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning
- 从少量真人示范自动生成双臂灵巧操作轨迹
- 仅用60个原始示范生成21,000条仿真数据
- 支持真实机器人部署，适用于双臂灵巧操作研究
通过人类示范进行模仿学习是教会机器人操作技能的有效方法。但数据获取是该范式广泛应用的主要瓶颈，因涉及大量成本和人力。对双臂灵巧机器人（如人形机器人）的模仿学习兴趣日益增长，然而由于需同时控制多臂与多指手，数据采集更为困难。模拟环境中的自动化数据生成是一种有前景且可扩展的替代方案。为此，我们提出DexMimicGen，一个大规模自动化数据生成系统，可从少量人类示范中合成适合具灵巧手人形机器人的操作轨迹。我们构建了涵盖多种操作行为及协调需求的仿真环境。仅用60个原始示范，生成了21,000条演示数据，并研究了多种数据生成与策略学习决策对智能体性能的影响。最后，我们展示了一个真实-模拟-真实迁移流程，并在真实人形机器人上完成了物品分类任务部署。数据集、仿真环境与补充结果见https://dexmimicgen.github.io/
原文摘要 · Abstract (English)
Imitation learning from human demonstrations is an effective means to teach robots manipulation skills. But data acquisition is a major bottleneck in applying this paradigm more broadly, due to the amount of cost and human effort involved. There has been significant interest in imitation learning for bimanual dexterous robots, like humanoids. Unfortunately, data collection is even more challenging here due to the challenges of simultaneously controlling multiple arms and multi-fingered hands. Automated data generation in simulation is a compelling, scalable alternative to fuel this need for data. To this end, we introduce DexMimicGen, a large-scale automated data generation system that synthesizes trajectories from a handful of human demonstrations for humanoid robots with dexterous hands. We present a collection of simulation environments in the setting of bimanual dexterous manipulation, spanning a range of manipulation behaviors and different requirements for coordination among the two arms. We generate 21K demos across these tasks from just 60 source human demos and study the effect of several data generation and policy learning decisions on agent performance. Finally, we present a real-to-sim-to-real pipeline and deploy it on a real-world humanoid can sorting task. Generated datasets, simulation environments and additional results are at https://dexmimicgen.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。