分析大模型回答视觉问题时看哪里，揭示其关注区域与答案正确性的关系。
Where do Large Vision-Language Models Look at when Answering Questions?
- 通过改进热力图方法，定位模型生成答案时依赖的图像区域。
- 发现模型关注区域与答案正确性显著相关，且不同架构注意力模式差异明显。
- 适合对视觉语言模型可解释性、模型行为分析感兴趣的读者。
大型视觉语言模型（LVLMs）在视觉语言理解与推理任务中表现优异，但其视觉理解行为仍不清晰。核心问题是：模型在多大程度上依赖视觉输入？哪些图像区域影响其回答？由于模型结构复杂（如多编码器、多分辨率）和输出长度可变，解释其自由生成的答案极具挑战。本文扩展现有热力图可视化方法（如iGOS++），支持开放式的视觉问答任务。提出一种方法，筛选反映生成答案与输入图像关联性的视觉相关标记。我们在需依赖视觉信息的基准测试上，对前沿的LVLMs进行了全面分析。研究发现揭示了关注区域与答案正确性的关系、不同架构间的注意力差异，以及大语言模型规模对视觉理解的影响。代码与数据已公开于https://github.com/bytedance/LVLM_Interpretation。
原文摘要 · Abstract (English)
Large Vision-Language Models (LVLMs) have shown promising performance in vision-language understanding and reasoning tasks. However, their visual understanding behaviors remain underexplored. A fundamental question arises: to what extent do LVLMs rely on visual input, and which image regions contribute to their responses? It is non-trivial to interpret the free-form generation of LVLMs due to their complicated visual architecture (e.g., multiple encoders and multi-resolution) and variable-length outputs. In this paper, we extend existing heatmap visualization methods (e.g., iGOS++) to support LVLMs for open-ended visual question answering. We propose a method to select visually relevant tokens that reflect the relevance between generated answers and input image. Furthermore, we conduct a comprehensive analysis of state-of-the-art LVLMs on benchmarks designed to require visual information to answer. Our findings offer several insights into LVLM behavior, including the relationship between focus region and answer correctness, differences in visual attention across architectures, and the impact of LLM scale on visual understanding. The code and data are available at https://github.com/bytedance/LVLM_Interpretation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。