通过增强关键视觉令牌注意力，显著减少大模型图像描述中的幻觉现象。
PAINT: Paying Attention to INformed Tokens to Mitigate Hallucination in Large Vision-Language Model
- 识别局部与摘要两类关键视觉令牌，针对性提升其注意力权重。
- 在MSCOCO数据集上将幻觉率降低62.3%，同时保持描述准确性。
- 无需训练的即插即用框架，适合希望提升生成可靠性的人使用。
大型视觉语言模型在理解与描述视觉内容方面表现卓越，但在生成描述时常引入图像中不存在的对象或细节，即幻觉问题。本文分析Transformer层中自注意力模式，发现幻觉源于深层网络中视觉令牌注意力权重逐渐减弱。现有方法简单提升所有视觉令牌注意力，效果不佳。我们识别出两类关键视觉令牌：局部令牌编码图像中真实物体的具身信息，摘要令牌捕捉图像整体聚合表征。二者需不同强度的权重增强。为此提出PAINT（Paying Attention to INformed Tokens）框架，通过实验确定的阈值，选择性增强局部与摘要令牌的注意力权重。在MSCOCO图像描述任务上的评估表明，该方法相比基线模型将幻觉率降低高达62.3%，同时保持高准确率。代码已开源。
原文摘要 · Abstract (English)
Large Vision Language Models (LVLMs) have demonstrated remarkable capabilities in understanding and describing visual content, achieving state-of-the-art performance across various vision-language tasks. However, these models often generate descriptions containing objects or details that are absent in the input image, a phenomenon commonly known as hallucination. Our work investigates the key reasons behind this issue by analyzing the pattern of self-attention in transformer layers. We find that hallucinations often arise from the progressive weakening of attention weight to visual tokens in the deeper layers of the LLM. Some previous works naively boost the attention of all visual tokens to mitigate this issue, resulting in suboptimal hallucination reduction. To address this, we identify two critical sets of visual tokens that facilitate the transfer of visual information from the vision encoder to the LLM. Local tokens encode grounded information about objects present in an image, while summary tokens capture the overall aggregated representation of the image. Importantly, these two sets of tokens require different levels of weight enhancement. To this end, we propose \textbf{PAINT} (\textbf{P}aying \textbf{A}ttention to \textbf{IN}formed \textbf{T}okens), a plug-and-play framework that intervenes in the self-attention mechanism of the LLM, selectively boosting the attention weights of local and summary tokens with experimentally learned margins. Evaluation on the MSCOCO image captioning dataset demonstrate that our approach reduces hallucination rates by up to 62.3\% compared to baseline models while maintaining accuracy. Code is available at \href{https://github.com/hasanar1f/PAINT}{https://github.com/hasanar1f/PAINT}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。