提出一种检测视觉语言模型幻觉回答的新方法，提升关键场景下的可靠性。
Detect Before You Leap: Mirage Detection in Vision-Language Models
- 通过分析图像块在各层的语义对齐变化，识别视觉证据缺失情况。
- 在10个领域1.9万样本上，将幻觉率从57%降至6.4%，准确率达88.4%。
- 适用于医疗、文档等高风险场景，帮助系统主动拒绝不可靠回答。
视觉语言模型（VLMs）在缺乏视觉证据、图像空白或无关时仍可能生成自信的视觉答案，这种现象被称为‘幻觉’（mirage），尤其在医疗和文档问答中危险。本文研究预发布幻觉检测：在答案输出前判断是否应释放。提出文本条件层内对齐（TC-LIA）方法，探测CLIP ViT-H/14编码器各层的图像块-文本对齐轨迹。通过投影各层图像块到最终嵌入空间，测量其与问题嵌入的相似性，追踪问题相关视觉证据是否出现。结合最终图像-文本余弦相似度、后期层top-k对齐、早期到晚期增益及层间斜率等特征，融合像素统计、零样本领域路由和结构化自评估，构建集成分类器。在涵盖医学影像、文档理解、场景文字等10个领域的19,004个样本上测试，覆盖四种输入条件：相关、同域无关、跨域无关、空白/噪声。14个VLM基线幻觉率57.3%-75.0%。TC-LIA单独使用可将幻觉率降至7.5%，准确率83.5%；集成后性能进一步提升至84.5%-88.4%准确率，幻觉率5.7%-7.2%。最佳结果来自Qwen2.5-VL-72B，达88.4%准确率，幻觉率6.4%。
原文摘要 · Abstract (English)
Vision-language models (VLMs) can produce confident visual answers even when the required visual evidence is missing, blank, or unrelated to the question. This failure mode, recently described as mirage (Asadi et al., 2026), is especially concerning in medical and document VQA, where visually ungrounded answers may be mistaken for image-based evidence. We study pre-release mirage detection: given an image-question pair, determine whether a VLM's answer should be released or the system should abstain before the answer reaches the user. We propose Text-Conditioned Layer-wise Internal Alignment (TC-LIA), a model-agnostic method that probes patch-token representations across the layers of a CLIP ViT-H/14 vision encoder. The key idea is to project layer-wise image patch tokens into the final CLIP embedding space and measure their similarity with the question embedding, tracking whether question-relevant visual evidence emerges across vision layers. TC-LIA summarizes this trajectory using final image-text cosine similarity, late-layer top-k patch-text alignment, early-to-late gain, and layer-wise slope. These features are combined with pixel-statistic-based blank/noise detection, zero-shot domain routing, and structured VLM self-assessment in an ensemble classifier. We evaluate on 19,004 samples spanning ten VQA domains, including medical imaging, document understanding, scene text, reasoning, and video understanding, with four input conditions: RELATED, IN-DOMAIN UNRELATED, OUT-OF-DOMAIN UNRELATED, and BLANK/NOISE. Across fourteen VLMs, base prompt mirage rates range from 57.3-75.0%. TC-LIA alone reduces the mirage rate to 7.5% at 83.5% accuracy; ensemble fusion further improves performance to 84.5-88.4% accuracy with 5.7-7.2% mirage rates. The best result, obtained with Qwen2.5-VL-72B, achieves 88.4% accuracy and a 6.4% mirage rate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。