arXiv:2409.14674cs.ROcs.CL2024-09ICRA被引 51

用语言引导的故障恢复机制,让机器人学会自己纠错。

RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learning

论文配图:RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learning
图 1 · 摘自论文原文
  • 自动生成带恢复轨迹和细粒度语言标注的数据
  • 在真实与仿真环境中超越现有最优模型表现
  • 适合需要自主纠错能力的复杂机械臂任务

由于缺乏自我恢复机制以及简单语言指令对机器人动作的指导能力有限,开发鲁棒且可纠正的视觉-运动策略在机器人操作中面临挑战。为此,我们提出一种可扩展的数据生成流水线,自动为专家演示添加故障恢复轨迹和细粒度语言标注以用于训练。随后,我们引入一种名为富语言引导故障恢复(RACER)的监督-执行框架,将故障恢复数据与丰富语言描述结合,提升机器人控制能力。RACER采用视觉-语言模型(VLM)作为在线监督者,提供详细的语言指导以修正错误并执行任务;同时使用语言条件化的视觉-运动策略作为执行者,预测下一步动作。实验结果表明,RACER在RLbench上的多种评估设置下均优于当前最先进的机器人视角变换器(RVT),包括标准长时序任务、动态目标变更任务及零样本未见任务,在模拟环境和真实世界中均表现出色。视频与代码已公开:https://rich-language-failure-recovery.github.io。

原文摘要 · Abstract (English)

Developing robust and correctable visuomotor policies for robotic manipulation is challenging due to the lack of self-recovery mechanisms from failures and the limitations of simple language instructions in guiding robot actions. To address these issues, we propose a scalable data generation pipeline that automatically augments expert demonstrations with failure recovery trajectories and fine-grained language annotations for training. We then introduce Rich languAge-guided failure reCovERy (RACER), a supervisor-actor framework, which combines failure recovery data with rich language descriptions to enhance robot control. RACER features a vision-language model (VLM) that acts as an online supervisor, providing detailed language guidance for error correction and task execution, and a language-conditioned visuomotor policy as an actor to predict the next actions. Our experimental results show that RACER outperforms the state-of-the-art Robotic View Transformer (RVT) on RLbench across various evaluation settings, including standard long-horizon tasks, dynamic goal-change tasks and zero-shot unseen tasks, achieving superior performance in both simulated and real world environments. Videos and code are available at: https://rich-language-failure-recovery.github.io.

机器人控制语言引导故障恢复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。