通过自适应增强生成文本注意力,有效减少视觉语言模型幻觉。
AdaIAT: Adaptively Increasing Attention to Generated Text to Alleviate Hallucinations in LVLM
- 利用生成文本中的视觉信息动态调整注意力权重。
- 在LLaVA-1.5上使幻觉率降低35.8%和37.1%。
- 适合关注幻觉问题且追求生成质量的开发者与研究者。
幻觉是当前大型视觉语言模型(LVLMs)发展与应用的主要障碍。为缓解幻觉,一种直观有效的方法是在推理时直接增加图像标记的注意力权重。尽管该方法显著降低幻觉率,但常导致描述重复。我们首先分析注意力模式,发现真实物体标记对生成文本的注意力高于幻觉标记。这一现象启发我们利用包含指令相关视觉信息与上下文知识的生成文本,以减轻幻觉并保持语言连贯性。因此提出注意力生成文本(IAT),并验证其可显著降低幻觉率且避免重复。为进一步防止盲目放大损害模型原有预测能力,我们提出自适应IAT(AdaIAT),采用分层阈值控制干预时机,并针对各注意力头特性精细调节放大强度。分析与实验表明AdaIAT有效:多个LVLM测试结果均显示,其在降低幻觉率(如在LLaVA-1.5上$C_S$和$C_I$分别下降35.8%和37.1%)的同时,保持语言性能与预测能力,实现良好权衡。
原文摘要 · Abstract (English)
Hallucination has been a significant impediment to the development and application of current Large Vision-Language Models (LVLMs). To mitigate hallucinations, one intuitive and effective way is to directly increase attention weights to image tokens during inference. Although this effectively reduces the hallucination rate, it often induces repetitive descriptions. To address this, we first conduct an analysis of attention patterns and reveal that real object tokens tend to assign higher attention to the generated text than hallucinated ones. This inspires us to leverage the generated text, which contains instruction-related visual information and contextual knowledge, to alleviate hallucinations while maintaining linguistic coherence. We therefore propose Attention to Generated Text (IAT) and demonstrate that it significantly reduces the hallucination rate while avoiding repetitive descriptions. To prevent naive amplification from impairing the inherent prediction capabilities of LVLMs, we further explore Adaptive IAT (AdaIAT) that employs a layer-wise threshold to control intervention time and fine-grained amplification magnitude tailored to the characteristics of each attention head. Both analysis and experiments demonstrate the effectiveness of AdaIAT. Results of several LVLMs show that AdaIAT effectively alleviates hallucination (reducing hallucination rates $C_S$ and $C_I$ on LLaVA-1.5 by 35.8% and 37.1%, respectively) while preserving linguistic performance and prediction capability, achieving an attractive trade-off.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。