让机器人用大模型+工具自动发现、解释并修复动作问题
RAIDER: Tool-Equipped Large Language Model Agent for Robotic Action Issue Detection, Explanation and Recovery
- 用'查证-提问-处理'流程动态生成问题并选工具
- 在模拟家庭环境里检测修复成功率超传统方法
- 适合需要人机协作的现实机器人任务
随着机器人在动态人本环境中应用增多，提升其对动作问题的检测、解释与恢复能力变得至关重要。传统基于模型和数据驱动的方法适应性差，而灵活的生成式AI又难以与真实世界约束对齐。我们提出RAIDER，一种将大语言模型（LLMs）与具身工具结合的新型智能体，实现自适应、高效的故障检测与解释。通过独特的“查证-提问-处理”流程，RAIDER动态生成上下文感知的前置条件问题，并选择合适工具进行信息获取，实现精准信息收集。在模拟家庭环境中的实验表明，其表现优于依赖预设模型、完整场景描述或独立训练模型的方法。此外，其解释能力显著提升了恢复成功率，包括需人工介入的情况。模块化架构搭配自校正机制，使其可轻松适配多种场景，已在真实人机协助任务中验证。这展示了RAIDER作为通用代理型AI在机器人问题检测与解释中的潜力，同时解决了生成式AI在具身智能体中落地的对齐难题。
原文摘要 · Abstract (English)
As robots increasingly operate in dynamic human-centric environments, improving their ability to detect, explain, and recover from action-related issues becomes crucial. Traditional model-based and data-driven techniques lack adaptability, while more flexible generative AI methods struggle with grounding extracted information to real-world constraints. We introduce RAIDER, a novel agent that integrates Large Language Models (LLMs) with grounded tools for adaptable and efficient issue detection and explanation. Using a unique "Ground, Ask&Answer, Issue" procedure, RAIDER dynamically generates context-aware precondition questions and selects appropriate tools for resolution, achieving targeted information gathering. Our results within a simulated household environment surpass methods relying on predefined models, full scene descriptions, or standalone trained models. Additionally, RAIDER's explanations enhance recovery success, including cases requiring human interaction. Its modular architecture, featuring self-correction mechanisms, enables straightforward adaptation to diverse scenarios, as demonstrated in a real-world human-assistive task. This showcases RAIDER's potential as a versatile agentic AI solution for robotic issue detection and explanation, while addressing the problem of grounding generative AI for its effective application in embodied agents. Project website: https://eurecat.github.io/raider-llmagent/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。