发现视觉语言模型‘过度思考’是幻觉根源，通过分析推理过程提升检测准确率。
Overthinking Causes Hallucination: Tracing Confounder Propagation in Vision Language Models
- 分析模型中间层推理过程，发现幻觉源于错误假设的持续传播
- 提出‘过度思考得分’指标，78.9% F1在MSCOCO上表现优异
- 适合关注模型可解释性与幻觉检测的研究者使用
视觉语言模型常产生不存在的物体幻觉。检测幻觉如同识别欺骗：仅看最终输出不足，需考察推理过程。现有方法多依赖最终层信号，注意力法假设幻觉词注意力低，熵值法依赖最终步骤的不确定性。我们的分析显示相反：幻觉对象可能因上下文先验而出现峰值注意力；模型在中间层已提前收敛至错误假设，故最终表达高度自信。关键在于模型的思维过程而非最终输出。通过探测解码器各层，我们发现此前未被重视的行为——‘过度思考’：模型在多层中反复修正物体假设，最终锁定错误答案。一旦模型陷入有偏假设，该偏差将贯穿后续层，导致幻觉。为此，我们提出‘过度思考得分’，衡量模型在多层中持有的竞争假设数量及其不稳定性。该得分显著提升幻觉检测效果：在MSCOCO上达78.9% F1，在AMBER上达71.58%。
原文摘要 · Abstract (English)
Vision Language models (VLMs) often hallucinate non-existent objects. Detecting hallucination is analogous to detecting deception: a single final statement is insufficient, one must examine the underlying reasoning process. Yet existing detectors rely mostly on final-layer signals. Attention-based methods assume hallucinated tokens exhibit low attention, while entropy-based ones use final-step uncertainty. Our analysis reveals the opposite: hallucinated objects can exhibit peaked attention due to contextual priors; and models often express high confidence because intermediate layers have already converged to an incorrect hypothesis. We show that the key to hallucination detection lies within the model's thought process, not its final output. By probing decoder layers, we uncover a previously overlooked behavior, overthinking: models repeatedly revise object hypotheses across layers before committing to an incorrect answer. Once the model latches onto a confounded hypothesis, it can propagate through subsequent layers, ultimately causing hallucination. To capture this behavior, we introduce the Overthinking Score, a metric to measure how many competing hypotheses the model entertains and how unstable these hypotheses are across layers. This score significantly improves hallucination detection: 78.9% F1 on MSCOCO and 71.58% on AMBER.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。