让机器人学会自己诊断并修复操作失败,提升真实场景下的可靠性。
RoboFAC: A Comprehensive Framework for Robotic Failure Analysis and Correction
- 构建了含9440条错误轨迹的大规模故障数据集,分类53种故障类型。
- 故障分析准确率比GPT-4o高34.1%,在真实任务中提升29.1%成功率。
- 轻量模型可本地部署,适合实际机器人系统快速响应故障。
视觉-语言-动作(VLA)模型通过将自然语言指令和视觉观察转化为控制动作,推动了机器人操作的发展。然而,现有VLA主要基于成功示范训练,缺乏对故障诊断与恢复的结构化监督,限制了其在开放世界中的鲁棒性。为此,我们提出机器人故障分析与纠正框架(RoboFAC)。构建了一个大规模以故障为中心的数据集,包含9,440条错误操作轨迹和78,623个问答对,覆盖53个仿真与真实场景,故障类型系统化分类。基于该数据集,开发了一种轻量级多模态模型,专用于任务理解、故障分析与纠正,支持高效本地部署且性能媲美大型专有模型。实验表明,RoboFAC在故障分析准确率上比GPT-4o高出34.1%。进一步将其作为外部监督器集成至真实VLA控制流程,在四个任务中实现29.1%的相对性能提升,同时显著降低延迟。结果证明,RoboFAC可系统化实现故障诊断与恢复,显著增强VLA的纠错能力。模型与数据集已公开于 https://github.com/MINT-SJTU/RoboFAC。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models have recently advanced robotic manipulation by translating natural-language instructions and visual observations into control actions. However, existing VLAs are primarily trained on successful expert demonstrations and lack structured supervision for failure diagnosis and recovery, limiting robustness in open-world scenarios. To address this limitation, we propose the Robotic Failure Analysis and Correction (RoboFAC) framework. We construct a large-scale failure-centric dataset comprising 9,440 erroneous manipulation trajectories and 78,623 QA pairs across 53 scenes in both simulation and real-world environments, with systematically categorized failure types. Leveraging this dataset, we develop a lightweight multimodal model specialized for task understanding, failure analysis, and failure correction, enabling efficient local deployment while remaining competitive with large proprietary models. Experimental results demonstrate that RoboFAC achieves a 34.1% higher failure analysis accuracy compared to GPT-4o. Furthermore, we integrated RoboFAC as an external supervisor in a real-world VLA control pipeline, yielding a 29.1% relative improvement across four tasks while significantly reducing latency relative to GPT-4o. These results demonstrate that RoboFAC enables systematic failure diagnosis and recovery, significantly enhancing VLA recovery capabilities. Our model and dataset are publicly available at https://github.com/MINT-SJTU/RoboFAC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。