用世界模型自动合成纠错数据,让机器人少犯错、少依赖真人示范。
WM-DAgger: Enabling Efficient Data Aggregation for Imitation Learning with World Models

- 用世界模型生成应对异常状态的恢复动作,避免错误累积。
- 仅用5次示范就实现93.3%成功率,显著提升机器人任务成功率。
- 适合少样本场景下的机器人操控,尤其适合无法频繁请人指导的场合。
模仿学习在训练机器人策略时表现强大,但受限于误差累积:微小策略偏差可能导致机器人进入训练集中未见的分布外(OOD)状态,从而引发更大错误并最终失败。尽管数据聚合(DAgger)框架试图缓解此问题,但其依赖持续的人类参与严重制约了可扩展性。本文提出WM-DAgger,一种无需人类干预的高效数据聚合框架,利用世界模型合成分布外状态下的恢复数据。针对眼手机器人抓取任务且仅有少量示范的情况,为避免生成误导性数据并克服世界模型固有的幻觉问题,本框架引入两个关键机制:(1) 纠正动作生成模块,生成任务导向的恢复动作以防止错误监督;(2) 一致性引导过滤模块,通过将合成轨迹终点帧与真实专家示范帧对齐,剔除物理上不合理的轨迹。我们在多个真实机器人任务上进行了充分验证,结果表明该方法显著提升成功率,在仅五次示范下软袋推移任务达到93.3%的成功率。源代码已公开于https://github.com/czs12354-xxdbd/WM-Dagger。
原文摘要 · Abstract (English)
Imitation learning is a powerful paradigm for training robotic policies, yet its performance is limited by compounding errors: minor policy inaccuracies could drive robots into unseen out-of-distribution (OOD) states in the training set, where the policy could generate even bigger errors, leading to eventual failures. While the Data Aggregation (DAgger) framework tries to address this issue, its reliance on continuous human involvement severely limits scalability. In this paper, we propose WM-DAgger, an efficient data aggregation framework that leverages World Models to synthesize OOD recovery data without requiring human involvement. Specifically, we focus on manipulation tasks with an eye-in-hand robotic arm and only few-shot demonstrations. To avoid synthesizing misleading data and overcome the hallucination issues inherent to World Models, our framework introduces two key mechanisms: (1) a Corrective Action Synthesis Module that generates task-oriented recovery actions to prevent misleading supervision, and (2) a Consistency-Guided Filtering Module that discards physically implausible trajectories by anchoring terminal synthesized frames to corresponding real frames in expert demonstrations. We extensively validate WM-DAgger on multiple real-world robotic tasks. Results that our method significantly improves success rates, achieving a 93.3\% success rate in soft bag pushing with only five demonstrations. The source code is publicly available at https://github.com/czs12354-xxdbd/WM-Dagger.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。