发现视觉模型幻觉源于注意力弱化,提出梯度感知修复方法。
Hallucination Begins Where Saliency Drops
- 融合注意力与梯度信号,量化输出词的视觉依存强度。
- 低显著性词导致幻觉,新方法使幻觉率显著下降。
- 适合关注大模型可信性的研究者与开发者使用。
近期研究通过分析大视觉语言模型(LVLMs)中的注意力动态来检测幻觉,但现有方法仅依赖前向传播的注意力模式,忽视了揭示标记影响传播的梯度信号,难以可靠区分幻觉与事实性输出。为此,我们提出 LVLMs-Saliency:一种基于梯度的诊断框架,通过融合注意力权重与输入梯度,量化每个输出标记的视觉接地强度。分析发现,当先前输出标记对下一个标记的预测显著性较低时,幻觉频繁出现,表明上下文记忆失效。基于此,我们提出双机制推理时框架:(1) 显著性引导拒绝采样(SGRS),在自回归解码中动态过滤显著性低于上下文自适应阈值的候选词,防止破坏连贯性的词进入序列;(2) 局部连贯性强化(LocoRE),一个轻量级、即插即用模块,增强当前词到最近前驱词的注意力,主动对抗上下文遗忘行为。在多个 LVLMs 上的实验证明,该方法显著降低幻觉率,同时保持流畅性和任务性能,提供了一种鲁棒且可解释的提升模型可靠性方案。代码见:https://github.com/zhangbaijin/LVLMs-Saliency
原文摘要 · Abstract (English)
Recent studies have examined attention dynamics in large vision-language models (LVLMs) to detect hallucinations. However, existing approaches remain limited in reliably distinguishing hallucinated from factually grounded outputs, as they rely solely on forward-pass attention patterns and neglect gradient-based signals that reveal how token influence propagates through the network. To bridge this gap, we introduce LVLMs-Saliency, a gradient-aware diagnostic framework that quantifies the visual grounding strength of each output token by fusing attention weights with their input gradients. Our analysis uncovers a decisive pattern: hallucinations frequently arise when preceding output tokens exhibit low saliency toward the prediction of the next token, signaling a breakdown in contextual memory retention. Leveraging this insight, we propose a dual-mechanism inference-time framework to mitigate hallucinations: (1) Saliency-Guided Rejection Sampling (SGRS), which dynamically filters candidate tokens during autoregressive decoding by rejecting those whose saliency falls below a context-adaptive threshold, thereby preventing coherence-breaking tokens from entering the output sequence; and (2) Local Coherence Reinforcement (LocoRE), a lightweight, plug-and-play module that strengthens attention from the current token to its most recent predecessors, actively counteracting the contextual forgetting behavior identified by LVLMs-Saliency. Extensive experiments across multiple LVLMs demonstrate that our method significantly reduces hallucination rates while preserving fluency and task performance, offering a robust and interpretable solution for enhancing model reliability. Code is available at: https://github.com/zhangbaijin/LVLMs-Saliency
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。