通过检查模型回答中的视觉证据，自动修复因表达不清导致的错误。
SAVER: Selective Auditing of Verbal Evidence for Error Recovery in VLM Change Reasoning
- 检测模型输出中是否包含物体名称、颜色、位置等明确证据。
- 在缺乏或矛盾证据时触发结构化重提示，最高提升25.8%准确率。
- 轻量级规则方法适配多模型，无需重新训练，适合实用系统优化。
视觉语言模型（VLM）在视觉变化推理任务中常出现错误，即使其视觉编码器已具备足够信息。我们观察到，正确输出通常包含支持变化结论的明确语言证据（如物体名称、颜色、空间位置），而错误输出往往缺乏此类证据。为此，提出SAVER（选择性言语证据审计），一种轻量级、基于规则的方法：解析VLM输出中的证据，并在证据缺失或不一致时触发结构化重提示。在三个变化检测基准和四种VLM上，该方法显著提升了因模型未能准确表达所见内容（表达失败）导致的错误场景下的准确率，最大提升达+25.8%（在CLEVR-Change上）。此外，利用大语言模型单次调用生成证据模式，效果可媲美人工调优的门控机制。消融实验表明，提升主要来自证据门控机制，而非重提示本身。
原文摘要 · Abstract (English)
Vision-language models (VLMs) frequently fail at visual change reasoning, even when their vision encoders contain sufficient information. We observe that correct VLM outputs tend to contain explicit verbal evidence (object names, colors, spatial locations) that supports the claimed change, while incorrect outputs often lack such evidence. We propose SAVER (Selective Auditing of Verbal Evidence for Error Recovery), a lightweight, rule-based method that parses VLM responses for this evidence and triggers structured reprompting only when evidence is missing or inconsistent. Across three change detection benchmarks and four VLMs, SAVER significantly improves accuracy on tasks where errors stem from the model failing to articulate what it saw (expression failures), with gains up to +25.8% on CLEVR-Change. The evidence patterns can also be generated by an LLM in a single call, matching the hand-tuned gate on CLEVR-Change. Ablation experiments confirm that the evidence gate, not reprompting alone, drives the improvement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。