arXiv:2606.17389cs.CVcs.AI2026-06中稿 · ICLR

视觉注意力不能判断模型是否胡说,生成过程的自一致性才是关键。

Visuals Lie, Consistency Speaks: Disentangling Spatial Attention from Reliability in Vision-Language Models

论文配图:Visuals Lie, Consistency Speaks: Disentangling Spatial Attention from Reliability in Vision-Language Models
图 1 · 摘自论文原文
  • 用注意力分布和空间熵量化视觉编码器的注视模式
  • 发现注意力与准确率几乎无关(相关性≈0.001)
  • 自一致性是预测可靠性的主要指标,适合调试模型

多模态大模型作为推理代理的使用日益广泛,其可靠性——即模型何时可能产生幻觉——变得至关重要。传统观点认为,注意力集中在相关区域意味着可信回答,分散则表示困惑。我们通过VLM可靠性探测器(VRP)对主流视觉语言模型进行跨架构研究,引入结构化注意力指标:聚类数(C_k)和空间熵(H_s),追踪其在各层的演化(Delta H_s)。结果揭示‘符号脱钩’现象:模型早期锁定视觉特征后,注意力迅速扩散,导致早期感知与最终生成脱节。相反,我们发现‘聚类失败’:空间注意力与准确率几乎无关(相关性≈0.001)。可靠性实质上是生成动态与内部状态分布的结果。自一致性(不同推理路径的一致率)是预测真实性的主导因素(相关性=0.429)。扩大因果干预显示显著架构差异:LLaVA在晚期阶段形成脆弱瓶颈,而PaliGemma与Qwen2-VL则全局分布可靠性,即使破坏50%以上最具预测力的层仍保持鲁棒。当前VLM的可靠性信号与视觉定位图无关,应从生成时动态与隐藏状态探针中推断。

原文摘要 · Abstract (English)

Multimodal Foundation Models are increasingly used as reasoning agents, making reliability, knowing when a model may hallucinate, critical. A common intuition, which we call the Attention-Confidence Assumption, holds that reliability follows from "structural" visual perception: tight attention on relevant regions should signal a trustworthy answer, while scattered attention signals confusion. We challenge this through the VLM Reliability Probe (VRP), a systematic cross-family study of reliability signals in contemporary Vision-Language Models (VLMs). We introduce structural-attention metrics, cluster counts (C_k) and spatial entropy (H_s), to quantify the visual encoder's gaze, and track its evolution (Delta H_s) across layers. This reveals a "Symbolic Detachment": models often "Early Lock" visual features only to diffuse attention later, severing early perception from final generation. Contrary to the grounding hypothesis, we find a "Cluster Failure": spatial attention has near-zero correlation (R approx 0.001) with accuracy. Instead, reliability is a phenomenon of generation dynamics and internal-state distributions. Self-Consistency, the agreement rate across sampled reasoning paths, is the dominant predictor of truth (R = 0.429). Scaling causal interventions exposes a sharp architectural divergence: LLaVA locks its prediction in a fragile late-stage bottleneck, whereas PaliGemma and Qwen2-VL distribute reliability globally, staying resilient even when ~50% or more of their most predictive layer is destroyed. For current VLMs, reliability signals are detached from visual grounding maps and are best inferred from generation-time dynamics and hidden-state probes.

视觉语言模型可靠性自一致性注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。