机器人通过自生成数据持续优化视觉规划能力,无需人类示范也能提升性能。
Self-Improving Loops for Visual Robotic Planning
- 用自收集轨迹迭代更新视频模型,实现在线自我改进
- 在多个未见过的任务上,性能随迭代持续提升
- 无需人工奖励或专家示范,比其他方法更高效
基于专家演示训练的视频生成模型已被用作高性能文本条件视觉规划器,用于解决机器人任务。然而,泛化到未见任务仍具挑战。尽管借助大规模离线视频数据可提升泛化能力,但在经验驱动时代,我们致力于设计能从自收集行为中持续改进的智能体。为此,本文提出视觉机器人规划的自改进循环(SILVR),即域内视频模型基于自产轨迹迭代更新,逐步提升特定任务表现。我们在MetaWorld系列任务及真实机械臂上的两项操作任务中验证了SILVR,发现其在初始训练未见过的新任务上,性能随多轮迭代持续提升。实验表明,SILVR无需人工提供的奖励函数或专家级演示,且在性能与样本效率方面优于其他利用在线经验的方法。
原文摘要 · Abstract (English)
Video generative models trained on expert demonstrations have been utilized as performant text-conditioned visual planners for solving robotic tasks. However, generalization to unseen tasks remains a challenge. Whereas improved generalization may be facilitated by leveraging learned prior knowledge from additional pre-collected offline data sources, such as web-scale video datasets, in the era of experience we aim to design agents that can continuously improve in an online manner from self-collected behaviors. In this work we thus propose the Self-Improving Loops for Visual Robotic Planning (SILVR), where an in-domain video model iteratively updates itself on self-produced trajectories, and steadily improves its performance for a specified task of interest. We apply SILVR to a diverse suite of MetaWorld tasks, as well as two manipulation tasks on a real robot arm, and find that performance improvements continuously emerge over multiple iterations for novel tasks unseen during initial in-domain video model training. We demonstrate that SILVR is robust in the absence of human-provided ground-truth reward functions or expert-quality demonstrations, and is preferable to alternate approaches that utilize online experience in terms of performance and sample efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。