用t分布过滤大模型幻觉,让机器人更可靠地模仿人类操作。
Robust Instant Policy: Leveraging Student's t-Regression Model for Robust In-context Imitation Learning of Robot Manipulation
- 用学生t分布聚合多个候选轨迹,抑制大模型生成的异常动作。
- 在模拟和真实环境中,任务成功率提升至少26%。
- 特别适合数据少的日常任务,提升机器人学习鲁棒性。
模仿学习(IL)旨在通过观察少量人类示范,让机器人自主完成任务。近期的上下文模仿学习(In-Context IL)利用现成的大语言模型(LLMs)作为即时策略,从少数示范中理解上下文并执行新任务,无需大规模示范训练模型。然而,其在机器人领域可靠性受制于大模型幻觉问题,时常生成偏离示范的劣质轨迹。为此,本文提出稳健的即时策略(RIP),采用学生t-回归模型,增强对即时策略幻觉轨迹的鲁棒性,实现可靠轨迹生成。具体而言,RIP从大语言模型生成多个候选机器人轨迹,并通过学生t分布进行聚合,该分布能有效忽略离群值(即幻觉动作),从而生成对抗幻觉的稳健轨迹。实验在模拟与真实环境均表明,RIP显著优于现有先进方法,任务成功率至少提升26%,尤其在低数据场景下表现优异。视频结果见:https://sites.google.com/view/robustinstantpolicy。
原文摘要 · Abstract (English)
Imitation learning (IL) aims to enable robots to perform tasks autonomously by observing a few human demonstrations. Recently, a variant of IL, called In-Context IL, utilized off-the-shelf large language models (LLMs) as instant policies that understand the context from a few given demonstrations to perform a new task, rather than explicitly updating network models with large-scale demonstrations. However, its reliability in the robotics domain is undermined by hallucination issues such as LLM-based instant policy, which occasionally generates poor trajectories that deviate from the given demonstrations. To alleviate this problem, we propose a new robust in-context imitation learning algorithm called the robust instant policy (RIP), which utilizes a Student's t-regression model to be robust against the hallucinated trajectories of instant policies to allow reliable trajectory generation. Specifically, RIP generates several candidate robot trajectories to complete a given task from an LLM and aggregates them using the Student's t-distribution, which is beneficial for ignoring outliers (i.e., hallucinations); thereby, a robust trajectory against hallucinations is generated. Our experiments, conducted in both simulated and real-world environments, show that RIP significantly outperforms state-of-the-art IL methods, with at least $26\%$ improvement in task success rates, particularly in low-data scenarios for everyday tasks. Video results available at https://sites.google.com/view/robustinstantpolicy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。