发现视觉语言模型中关键注意力头被无关词劫持,提出方法有效减少幻觉。
Vocabulary Hijacking in LVLMs: Unveiling Critical Attention Heads by Excluding Inert Tokens to Mitigate Hallucination

- 通过分析注意力机制,识别出导致幻觉的惰性视觉标记及其劫持锚点。
- 提出NHAR指标,量化关键注意力头对劫持的抵抗能力,提升准确性。
- 无需额外计算开销,可直接增强模型对视觉内容的关注,适合追求可靠性的应用者。
大型视觉语言模型(LVLMs)在多模态任务中取得显著进展,但其可靠性常受幻觉问题困扰——即生成与视觉输入矛盾的文本。现有研究多将错误归因于视觉注意力不足。本文通过逻辑探针分析注意力机制,揭示了一种称为“词汇劫持”的异常现象。我们发现某些特定视觉标记(称作惰性标记)过度吸引注意力。关键在于,这些标记在中间隐藏状态投影到词汇空间后,始终解码为一组固定且无关的词语(称作劫持锚点),显示出严格的语义坍缩。基于此语义刚性,我们提出劫持锚点定位方法(HABI),可精准识别这些惰性标记。为进一步量化影响,我们引入非劫持视觉注意力比(NHAR),用于识别对劫持具有鲁棒性的关键注意力头,这些头对事实准确性至关重要。基于上述发现,我们提出劫持感知视觉注意力增强(HAVAE),一种无需训练的干预策略,可选择性强化这些关键头对显著视觉内容的关注。在多个基准测试中的大量实验表明,HAVAE显著减轻了幻觉,且无额外计算开销,同时保持模型通用能力。代码已公开于 https://github.com/lab-klc/HAVAE。
原文摘要 · Abstract (English)
Large Vision-Language Models (LVLMs) have achieved remarkable progress in multimodal tasks, yet their reliability is persistently undermined by hallucinations-generating text that contradicts visual input. Recent studies often attribute these errors to inadequate visual attention. In this work, we analyze the attention mechanisms via the logit lens, uncovering a distinct anomaly we term Vocabulary Hijacking. We discover that specific visual tokens, defined as Inert Tokens, disproportionately attract attention. Crucially, when their intermediate hidden states are projected into the vocabulary space, they consistently decode to a fixed set of unrelated words (termed Hijacking Anchors) across layers, revealing a rigid semantic collapse. Leveraging this semantic rigidity, we propose Hijacking Anchor-Based Identification (HABI), a robust strategy to accurately localize these Inert Tokens. To quantify the impact of this phenomenon, we introduce the Non-Hijacked Visual Attention Ratio (NHAR), a novel metric designed to identify attention heads that remain resilient to hijacking and are critical for factual accuracy. Building on these insights, we propose Hijacking-Aware Visual Attention Enhancement (HAVAE), a training-free intervention that selectively strengthens the focus of these identified heads on salient visual content. Extensive experiments across multiple benchmarks demonstrate that HAVAE significantly mitigates hallucinations with no additional computational overhead, while preserving the model's general capabilities. Our code is publicly available at https://github.com/lab-klc/HAVAE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。