arXiv:2606.24633cs.RO2026-06

让机器人从有错误的示范中学习,通过重试行为识别并纠正失误。

Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation

论文配图:Beyond Monotonic Progress: Retry-Supervised Value Learning for Robot Imitation
图 1 · 摘自论文原文
  • 利用重试事件作为稀疏标注,学习对错误敏感的价值函数。
  • 在真实机器人操作任务中,比基于进度的基线方法更精细地评估执行质量。
  • 适合需要从含错示范中学习的机器人模仿学习场景。

人类示范中的错误和修正行为(如抓取不准、物体偏移、接触不稳、重复尝试)常被视为噪声或次优数据,但这些片段实际上提供了执行偏离理想路径及恢复可行性的重要线索。现有奖励与价值模型多依赖单调进展假设,难以捕捉局部执行错误与修正行为。本文提出ReTVL(Retry-Supervised Value Learning)框架,通过稀疏标注的重试关键帧,结合全局进度校准与局部成对偏好学习,构建对错误敏感的价值函数。该模型用于重加权示范片段,降低有害错误的影响,同时保留有用修正行为。真实机器人操作任务实验表明,ReTVL生成的价值估计比基于进度的基线更细粒度,并显著提升从不完美示范中学习的表现。

原文摘要 · Abstract (English)

Human demonstrations for robot imitation learning often contain mistakes and corrective behaviors, such as imprecise grasps, object misalignment, unstable contact, and repeated attempts. While these segments are commonly treated as noisy or suboptimal data, they provide valuable evidence about when execution deviates from a desirable path and how task feasibility can be restored. However, existing reward and value models often rely on monotonic progress assumptions, which capture coarse task advancement but may overlook local execution errors and corrective behaviors in imperfect demonstrations. In this work, we propose ReTVL (ReTry-Supervised Value Learning), a framework for learning mistake-sensitive value functions from mixed-quality robot demonstrations by leveraging retry events as sparse supervision. ReTVL captures the local degradation-and-recovery structure around mistakes by combining global progress calibration with local pairwise preference learning induced by sparsely annotated retry keypoints. The learned value model is then used to reweight demonstration chunks for downstream behavior cloning, reducing the influence of harmful execution errors while preserving useful corrective behaviors. Experiments on real-robot manipulation tasks show that ReTVL produces more fine-grained value estimates than progress-based baselines and improves imitation learning from imperfect demonstrations.

机器人模仿价值学习错误纠正

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。