用扩散模型修复带噪声的专家示范,提升模仿学习效果。
Restoring Noisy Demonstration for Imitation Learning With Diffusion Models
- 先筛选干净样本,再用条件扩散模型恢复噪声样本。
- 在机械臂、灵巧操作和行走任务中均优于现有方法。
- 对不同噪声类型和程度都有良好鲁棒性,适合真实数据。
模仿学习(IL)旨在从专家示范中学习策略,广泛应用于机器人控制等领域。现有方法多假设示范数据完美无瑕,但实际中常因人为误差或传感器/控制系统不精准导致数据含噪。为此,本文提出一种滤除-恢复框架,以充分利用存在噪声的示范数据。首先从示范中筛选出干净样本,再训练条件扩散模型来重建噪声样本。我们在机械臂操作、灵巧操作和运动控制等多个领域验证该框架及现有方法的表现。实验结果表明,所提框架在所有任务中均持续优于现有方法。消融实验证明各模块有效性,并展示框架对不同噪声类型与强度的鲁棒性。结果证实该框架在处理含噪离线示范数据方面具备实际应用价值。
原文摘要 · Abstract (English)
Imitation learning (IL) aims to learn a policy from expert demonstrations and has been applied to various applications. By learning from the expert policy, IL methods do not require environmental interactions or reward signals. However, most existing imitation learning algorithms assume perfect expert demonstrations, but expert demonstrations often contain imperfections caused by errors from human experts or sensor/control system inaccuracies. To address the above problems, this work proposes a filter-and-restore framework to best leverage expert demonstrations with inherent noise. Our proposed method first filters clean samples from the demonstrations and then learns conditional diffusion models to recover the noisy ones. We evaluate our proposed framework and existing methods in various domains, including robot arm manipulation, dexterous manipulation, and locomotion. The experiment results show that our proposed framework consistently outperforms existing methods across all the tasks. Ablation studies further validate the effectiveness of each component and demonstrate the framework's robustness to different noise types and levels. These results confirm the practical applicability of our framework to noisy offline demonstration data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。