用检索增强生成技术提升机器人故障识别准确率
Fail-RAG : A Retrieval Augmented Generation Informed Framework for Robot Failure Identification
- 基于RAG框架，通过图像与上下文相似性匹配故障数据库
- 相比现成视觉语言模型，故障检测准确率平均提升25个百分点
- 适合制造仓储场景中需要实时应对突发故障的智能机器人
工业自动化正因技术突破与社会变迁推动机器人发展：向通用型机器人、具身物理人工智能演进，且制造业面临日益严重的劳动力短缺。智能自主机器人不仅需按计划执行动作，还需应对意外事件。本研究聚焦仓库中用于物料搬运的机器人所面临的意外事件，将此类事件统称为故障，并开发方法以检测与机器人操作相关的故障。基于规则的检测方法可能因环境与任务动态变化导致失效。我们提出Fail-RAG，一种基于检索增强生成（RAG）的故障检测框架，将故障图像与上下文信息嵌入并查询故障数据库，通过计算相似性实现匹配。进一步利用视觉-语言模型（VLMs）根据指令模板分析故障并提供细节。在固定机械臂和移动操作臂上，针对多种常见仓储自动化任务，通过仿真与实物实验评估了Fail-RAG性能。结果表明，相比使用现成VLMs，Fail-RAG在五类机器人操作中的故障检测准确率平均提升25个百分点，证明其在真实场景下故障检测的有效性。
原文摘要 · Abstract (English)
Industry automation is witnessing an evolution in robotics driven by both technological breakthroughs and societal changes: progress towards generalist robots, embodied and physical artificial intelligence (AI), and increasing labor shortage in manufacturing.An intelligent autonomous robot needs to not only act according to planned motions but also react to any unexpected events. In this study, we focus on such unexpected events in warehouses where robots are used for material handling. Specifically, we refer to any unexpected events as failures and develop methods to detect robot operations related failures. Rule-based detection methods may break since the form of failures could change due to the dynamic nature of both environments and tasks. We propose 'Fail-RAG', a Retrieval Augmented Generation (RAG)-based failure detection framework where failure images and context information are embedded and queried against a failure database by calculating their similarities. Vision-Language Models (VLMs) are further used to analyze failures and provide details by following our instruction template. We evaluated the performance of Fail-RAG by conducting both simulation and physical experiments using fixed robot arms and a mobile manipulator for multiple tasks that are common in warehouse automation. Fail-RAG achieved 25 percentage point higher failure detection accuracy on average across five types of robot operations compared to using off-the-shelf VLMs, indicating its effectiveness for real-world failure detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。