不生成任何文字就能提前检测视觉语言模型的幻觉,提升安全性和效率。
HALP: Detecting Hallucinations in Vision-Language Models without Generating a Single Token
- 通过分析模型内部表示,无需生成文本即可预判幻觉风险。
- 在8个主流模型上最高达0.93的AUROC,证明预测有效。
- 轻量级探测器可实现早期拒绝回答,适合部署在高安全场景。
幻觉仍是视觉语言模型(VLMs)的持续挑战,常会描述不存在的对象或编造事实。现有检测方法通常在文本生成后进行,干预成本高且不及时。我们研究是否可通过单次前向传播,探测模型内部表征来预先预测幻觉风险。在涵盖多种视觉语言任务及八种现代VLMs(包括Llama-3.2-Vision、Gemma-3、Phi-4-VL和Qwen2.5-VL)的实验中,考察三类内部表示:(i) 未融合多模态的纯视觉特征,(ii) 文本解码器中的视觉标记表示,(iii) 生成前整合视觉与文本信息的查询标记表示。基于这些表示训练的探测器在无解码情况下表现出色,最高达0.93 AUROC(Gemma-3-12B、Phi-4-VL 5.6B、Molmo 7B)。多数模型中,晚期查询标记状态最具预测力;少数架构中,纯视觉或中层特征更优(如Qwen2.5-VL-7B使用纯视觉特征达~0.79 AUROC)。结果表明:(1) 幻觉风险可在生成前被检测,(2) 最具信息量的层与模态因架构而异,(3) 轻量级探测器有望实现早期拒答、选择性路由与自适应解码,从而提升安全性与效率。
原文摘要 · Abstract (English)
Hallucinations remain a persistent challenge for vision-language models (VLMs), which often describe nonexistent objects or fabricate facts. Existing detection methods typically operate after text generation, making intervention both costly and untimely. We investigate whether hallucination risk can instead be predicted before any token is generated by probing a model's internal representations in a single forward pass. Across a diverse set of vision-language tasks and eight modern VLMs, including Llama-3.2-Vision, Gemma-3, Phi-4-VL, and Qwen2.5-VL, we examine three families of internal representations: (i) visual-only features without multimodal fusion, (ii) vision-token representations within the text decoder, and (iii) query-token representations that integrate visual and textual information before generation. Probes trained on these representations achieve strong hallucination-detection performance without decoding, reaching up to 0.93 AUROC on Gemma-3-12B, Phi-4-VL 5.6B, and Molmo 7B. Late query-token states are the most predictive for most models, while visual or mid-layer features dominate in a few architectures (e.g., ~0.79 AUROC for Qwen2.5-VL-7B using visual-only features). These results demonstrate that (1) hallucination risk is detectable pre-generation, (2) the most informative layer and modality vary across architectures, and (3) lightweight probes have the potential to enable early abstention, selective routing, and adaptive decoding to improve both safety and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。