让视觉语言模型在推理时主动回看图像，减少幻觉并提升准确率。
Qwen Look Again: Guiding Vision-Language Reasoning Models to Re-attention Visual Information
- 引入视觉-文本反思机制，引导模型主动重新关注图像信息。
- 在多个视觉问答数据集上，准确率领先且幻觉率显著降低。
- 适合需要高可靠视觉推理的应用，如医疗、自动驾驶场景。
推理时的扩展推理能提升视觉语言模型（VLMs）性能，形成强大的视觉语言推理模型（VLRMs）。然而，长推理过程会稀释视觉标记，导致视觉信息关注度下降，可能引发幻觉。尽管文本独反思在语言模型中表现良好，但在VLMs中仍不足以抑制幻觉。为此，我们提出Qwen-LookAgain（Qwen-LA），通过引入视觉-文本反思机制，在推理过程中引导模型重新关注视觉信息以缓解幻觉。我们首先提出平衡反思策略优化（BRPO）的强化学习方法，使模型自主决定何时生成反思，并平衡反思次数与长度。进一步证明了随着推理推进，VLRMs对视觉标记的关注度逐渐下降，并证实反思阶段补充视觉信息可增强视觉注意力。因此，训练与推理中引入视觉标记复制（Visual Token COPY）和视觉标记路由（Visual Token ROUTE）机制，强制模型在视觉层面重新关注图像信息，克服纯文本反思的局限。在多个视觉问答数据集及幻觉评估指标上的实验表明，Qwen-LA在保持领先准确率的同时有效降低幻觉。代码已开源：https://github.com/Liar406/Look_Again
原文摘要 · Abstract (English)
Inference time scaling drives extended reasoning to enhance the performance of Vision-Language Models (VLMs), thus forming powerful Vision-Language Reasoning Models (VLRMs). However, long reasoning dilutes visual tokens, causing visual information to receive less attention and may trigger hallucinations. Although introducing text-only reflection processes shows promise in language models, we demonstrate that it is insufficient to suppress hallucinations in VLMs. To address this issue, we introduce Qwen-LookAgain (Qwen-LA), a novel VLRM designed to mitigate hallucinations by incorporating a vision-text reflection process that guides the model to re-attention visual information during reasoning. We first propose a reinforcement learning method Balanced Reflective Policy Optimization (BRPO), which guides the model to decide when to generate vision-text reflection on its own and balance the number and length of reflections. Then, we formally prove that VLRMs lose attention to visual tokens as reasoning progresses, and demonstrate that supplementing visual information during reflection enhances visual attention. Therefore, during training and inference, Visual Token COPY and Visual Token ROUTE are introduced to force the model to re-attention visual information at the visual level, addressing the limitations of text-only reflection. Experiments on multiple visual QA datasets and hallucination metrics indicate that Qwen-LA achieves leading accuracy performance while reducing hallucinations. Our code is available at: https://github.com/Liar406/Look_Again
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。