发现视觉语言模型的可解释性失效模式,助力安全改进。
Revealing Interpretable Failure Modes of VLMs

- 构建搜索框架REVELIO,系统定位由具体概念组合引发的失效
- 在自动驾驶与室内机器人中发现新漏洞,模型常误判障碍物或过度保守
- 提供可理解的失败原因,适合安全评估与模型优化团队使用
视觉语言模型(VLMs)因其强大的推理能力和少任务工程的泛化能力,正被广泛应用于安全关键场景。然而,在特定真实情境下可能产生灾难性失效,形成可识别的失败模式。本文提出REVELIO框架,用于系统揭示可解释的失败模式。我们将失败模式定义为由行人距离、恶劣天气等领域相关概念组成的组合,导致目标VLM持续错误行为。识别此类模式需遍历指数级的离散组合空间。为此,REVELIO结合两种搜索策略:多样性感知的束搜索以高效绘制失效图谱,以及基于高斯过程的汤普森采样策略实现对复杂失效模式的广泛探索。我们在自动驾驶与室内机器人领域应用该方法,发现了现有顶尖VLM中存在的此前未报告的脆弱性。在驾驶环境中,模型常表现出空间定位薄弱,无法正确识别主要障碍物,导致模拟碰撞建议;在室内任务中,模型要么遗漏安全风险,要么过度保守,引发误报并降低运行效率。通过揭示结构化且可解释的失败模式,REVELIO为提升VLM安全性提供了可操作的洞察。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) are increasingly used in safety-critical applications because of their broad reasoning capabilities and ability to generalize with minimal task-specific engineering. Despite these advantages, they can exhibit catastrophic failures in specific real-world situations, constituting failure modes. We introduce REVELIO, a framework for systematically uncovering interpretable failure modes in VLMs. We define a failure mode as a composition of interpretable, domain-relevant concepts-such as pedestrian proximity or adverse weather conditions-under which a target VLM consistently behaves incorrectly. Identifying such failures requires searching over an exponentially large discrete combinatorial space. To address this challenge, REVELIO combines two search procedures: a diversity-aware beam search that efficiently maps the failure landscape, and a Gaussian-process Thompson Sampling strategy that enables broader exploration of complex failure modes. We apply REVELIO to autonomous driving and indoor robotics domains, uncovering previously unreported vulnerabilities in state-of-the-art VLMs. In driving environments, the models often demonstrate weak spatial grounding and fail to account for major obstructions, leading to recommendations that would result in simulated crashes. In indoor robotics tasks, VLMs either miss safety hazards or behave excessively conservatively, producing false alarms and reducing operational efficiency. By identifying structured and interpretable failure modes, REVELIO offers actionable insights that can support targeted VLM safety improvements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。