让机器人主动精准要示范,减少试错次数。
DISEIL: Demonstration Distillation for Sample-Efficient Imitation Learning

- 根据失败时刻的几何特征聚类错误模式,自动定位需补充示范的位置。
- 在5个模拟任务中,用最少示范即达最高成功率,小样本下优势最明显。
- 适合想降低人工示范成本的机器人学习研究者,尤其关注高效模仿学习场景。
能从极少示范中学会新任务的机器人必须自行判断缺失能力,并精准请求所需帮助。交互式模仿学习让策略自主练习并在出错时求助专家。现有方法仅决定何时中断,而由触发中断的片段自行决定修正哪个错误、从何处开始示范。本文首次明确设计这两个决策:DISEIL(演示蒸馏的高效模仿学习)在策略首次不可靠的步骤标记失败,用几何描述符表示该时刻,并将失败归类为重复出现的错误模式。视觉-语言模型与语言模型分析选定模式并生成示范请求,任务约束库验证请求可行性后才消耗专家时间。无模型直接输出机器人动作。在5个带状态和图像观测的模拟任务中,仅调整专家请求内容,便在所有10种设置下取得最高平均保留成功率,其中1次并列,最小样本预算下差距最大。局限在于单轮练习、仿真环境、专家多为脚本化。长远目标是让学习器也跟踪已有示范覆盖范围,按请求成本比例向人类教师索取缺失行为。
原文摘要 · Abstract (English)
A robot that can be taught a new task from a handful of demonstrations has to work out for itself what it still cannot do, and then ask for exactly that. Interactive imitation learning takes a step in that direction by letting a policy practice on its own and calling an expert when it goes wrong. Existing methods decide when to interrupt the learner. A further 2 decisions are left to whichever episode happened to trigger the interruption: which failure to correct, and where the demonstration should start. This paper is a first attempt at making both of them deliberately. DISEIL (Demonstration dIstillation for Sample-Efficient Imitation Learning) marks each failed episode at the step where the policy first becomes unreliable, represents that moment with a geometric descriptor, and groups the failures into recurring failure modes. A vision-language model and a language model read the selected mode and write a request for the next demonstration, and a store of task constraints checks that the request can be carried out before any expert time is spent. No model produces a robot action. Across 5 simulated tasks under state and image observations, changing only what the expert is asked for gives the highest mean held-out success rate in all 10 settings, with a tie in 1, and the margin is widest at the smallest budget we tested. The scope is narrow: a single round of practice at a time, in simulation, with experts that are mostly scripted. The longer-term aim is a learner that also tracks what its demonstration set already covers, and that asks a human teacher for the missing behavior in proportion to the effort each request costs them.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。