提出轻量级框架VLA-FAIL,实时检测视觉语言动作模型的任务失败。
VLA-FAIL: Efficient Task Failure Detection for Finetuned Vision-Language-Action Models

- 基于最后一层特征偏差和动作块一致性,无需失败数据。
- 在真实与仿真环境中均实现早期可靠检测,优于复杂基线方法。
- 适合机器人操控中需高安全性的部署场景。
视觉-语言-动作模型(VLAs)在众多机器人操作任务中表现卓越,但在分布外场景下仍可能行为不可预测。因此,运行时失败检测对VLAs的安全实际部署至关重要。然而,现有失败检测器需昂贵的动作采样,依赖特定架构假设,或需失败回放数据。本文提出VLA-FAIL,一种轻量且通用的失败检测框架,结合两种新检测器,开销极小,无需失败数据。第一种为最后一层马氏距离(LLMD),通过测量最后一层特征相对于训练数据的词元级偏差来检测分布外状态;第二种为动作块一致性(ACC),利用滚动视界控制带来的时序重叠,当连续动作块不一致时判定失败。为权衡检测准确率与延迟,引入无阈值指标AUCPDT,联合评估精度、召回率与检测时间。通过大量真实世界与仿真实验,验证了LLMD与ACC捕捉互补失败模式,组合后可在多样任务中实现可靠且早期的失败检测,显著优于更复杂的基线方法。
原文摘要 · Abstract (English)
Vision-language-action models (VLAs) achieve state-of-the-art performance on many robotic manipulation tasks, yet they can still behave unpredictably in out-of-distribution scenarios. Runtime failure detection is therefore essential for the safe real-world deployment of VLAs. However, existing task failure detectors require computationally expensive action sampling, are based on architectural assumptions that limit their applicability to VLAs, or need access to failure rollouts. We propose VLA-FAIL, a lightweight and broadly applicable failure detection framework for VLAs that combines two novel failure detectors with minimal overhead, without requiring failure data. The first, last-layer Mahalanobis distance (LLMD), detects out-of-distribution states by measuring token-wise deviations in last-layer features relative to the training data. The second, action chunk consistency (ACC), exploits the temporal overlap induced by receding-horizon control and detects failures when consecutive action chunks become inconsistent. To capture the trade-off between detection accuracy and detection latency, we introduce AUCPDT, a threshold-independent metric that jointly evaluates precision, recall, and detection time. Through extensive real-world and simulation experiments, we demonstrate that LLMD and ACC capture complementary failure modes whose combination enables reliable and early failure detection across diverse tasks, frequently outperforming significantly more expensive baseline methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。