通过自迭代循环生成多样化机器人操作数据,提升真实场景泛化能力。
DexFlyWheel: A Scalable and Self-improving Data Generation Framework for Dexterous Manipulation
- 用模仿学习+残差强化学习构建闭环生成流程
- 生成超2000条多样演示,实测成功率81.9%
- 适合需要高泛化能力的复杂抓取任务研究者
灵巧操作对推动机器人在现实应用中的能力至关重要,但多样且高质量的数据集仍稀缺。现有数据采集方法或依赖人工遥控,或需大量人工工程设计,或生成数据多样性有限,限制了可扩展性和泛化能力。本文提出DexFlyWheel,一个可扩展的数据生成框架,采用自改进循环持续丰富数据多样性。从高效种子示范启动开始,通过迭代循环扩展数据集。每个循环遵循闭合回路流程:模仿学习(IL)从示范中提取类人行为,残差强化学习(RL)增强策略泛化性;所学策略在仿真中生成轨迹,经多环境与空间配置的数据增强后反馈至下一轮。经过多次迭代,形成自我强化的数据飞轮效应,生成覆盖多种场景的数据集,从而提升策略性能。实验表明,DexFlyWheel在四项挑战任务中生成超过2000条多样化示范。基于该数据集训练的策略在挑战测试集上平均成功率达81.9%,并通过数字孪生成功迁移到真实世界,在双臂搬运任务中实现78.3%的成功率。
原文摘要 · Abstract (English)
Dexterous manipulation is critical for advancing robot capabilities in real-world applications, yet diverse and high-quality datasets remain scarce. Existing data collection methods either rely on human teleoperation or require significant human engineering, or generate data with limited diversity, which restricts their scalability and generalization. In this paper, we introduce DexFlyWheel, a scalable data generation framework that employs a self-improving cycle to continuously enrich data diversity. Starting from efficient seed demonstrations warmup, DexFlyWheel expands the dataset through iterative cycles. Each cycle follows a closed-loop pipeline that integrates Imitation Learning (IL), residual Reinforcement Learning (RL), rollout trajectory collection, and data augmentation. Specifically, IL extracts human-like behaviors from demonstrations, and residual RL enhances policy generalization. The learned policy is then used to generate trajectories in simulation, which are further augmented across diverse environments and spatial configurations before being fed back into the next cycle. Over successive iterations, a self-improving data flywheel effect emerges, producing datasets that cover diverse scenarios and thereby scaling policy performance. Experimental results demonstrate that DexFlyWheel generates over 2,000 diverse demonstrations across four challenging tasks. Policies trained on our dataset achieve an average success rate of 81.9\% on the challenge test sets and successfully transfer to the real world through digital twin, achieving a 78.3\% success rate on dual-arm lift tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。