用自迭代推理提升机器人故障检测与原因分析能力
Self-Refining Vision Language Model for Robotic Failure Detection and Reasoning
- 通过多任务自精炼框架，结合历史输出生成推理链
- 故障检测率提升30%，推理准确率提高100%（基于LLM模糊匹配）
- 适用于真实场景中标注稀缺、故障多样化的机器人系统
故障推理对构建可靠可信的机器人系统至关重要。以往方法要么将故障推理视为封闭集分类问题，要么依赖大量人工标注。现实中的故障通常细微、组合复杂且难以枚举，而高质量推理标签获取成本高昂。为此，我们提出ARMOR：用于机器人故障检测与推理的自适应多轮多任务模型。将检测与推理建模为多任务自精炼过程，模型可迭代预测检测结果和自然语言推理，且推理依赖于历史输出。训练阶段，ARMOR融合大规模稀疏二值标签与小规模丰富推理标注，通过离线与在线模仿学习联合优化。推理时，生成多条精炼轨迹，并基于自置信度指标选择最可靠预测。在多种环境下的实验表明，相比先前方法，ARMOR在故障检测率上最高提升30%，推理准确率（通过LLM模糊匹配分数衡量）提升达100%，展现出对异构监督的鲁棒性及对预设故障模式之外的开放推理能力。更多可视化内容见官网：https://sites.google.com/utexas.edu/armor
原文摘要 · Abstract (English)
Reasoning about failures is crucial for building reliable and trustworthy robotic systems. Prior approaches either treat failure reasoning as a closed-set classification problem or assume access to ample human annotations. Failures in the real world are typically subtle, combinatorial, and difficult to enumerate, whereas rich reasoning labels are expensive to acquire. We address this problem by introducing ARMOR: Adaptive Round-based Multi-task mOdel for Robotic failure detection and reasoning. We formulate detection and reasoning as a multi-task self-refinement process, where the model iteratively predicts detection outcomes and natural language reasoning conditioned on past outputs. During training, ARMOR learns from heterogeneous supervision - large-scale sparse binary labels and small-scale rich reasoning annotations - optimized via a combination of offline and online imitation learning. At inference time, ARMOR generates multiple refinement trajectories and selects the most confident prediction via a self-certainty metric. Experiments across diverse environments show that ARMOR achieves state-of-the-art performance by improving over the previous approaches by up to 30% on failure detection rate and up to 100% in reasoning measured through LLM fuzzy match score, demonstrating robustness to heterogeneous supervision and open-ended reasoning beyond predefined failure modes. We provide dditional visualizations on our website: https://sites.google.com/utexas.edu/armor
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。