解析Llava视觉问答机制，揭示其类上下文学习特性
Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering
- 通过机制可解释性分析视觉问答过程
- 发现视觉特征在嵌入空间中高度可解释
- 工具可定位关键视觉区域，帮助理解幻觉现象
理解大语言模型（LLMs）的内在机制对改进模型设计至关重要。尽管已有研究深入揭示了文本型大模型的机制，多模态大模型（MLLMs）的机制仍鲜有探索。本文首次对首个多模态大模型Llava在视觉问答（VQA）任务中的机制进行系统分析，对比了其与文本问答（TQA）在颜色回答任务中的差异：1）VQA表现出与文本上下文学习相似的机制；2）将视觉嵌入投影到嵌入空间后，视觉特征具有显著可解释性；3）在视觉指令微调过程中，Llava增强了对应文本模型Vicuna的原有能力。基于上述发现，我们开发了一款可解释性工具，可帮助用户和研究者识别影响最终预测的关键视觉区域，辅助理解视觉幻觉。该方法相比现有手段更快速、更有效。代码已开源：https://github.com/zepingyu0512/llava-mechanism。
原文摘要 · Abstract (English)
Understanding the mechanisms behind Large Language Models (LLMs) is crucial for designing improved models and strategies. While recent studies have yielded valuable insights into the mechanisms of textual LLMs, the mechanisms of Multi-modal Large Language Models (MLLMs) remain underexplored. In this paper, we apply mechanistic interpretability methods to analyze the visual question answering (VQA) mechanisms in the first MLLM, Llava. We compare the mechanisms between VQA and textual QA (TQA) in color answering tasks and find that: a) VQA exhibits a mechanism similar to the in-context learning mechanism observed in TQA; b) the visual features exhibit significant interpretability when projecting the visual embeddings into the embedding space; and c) Llava enhances the existing capabilities of the corresponding textual LLM Vicuna during visual instruction tuning. Based on these findings, we develop an interpretability tool to help users and researchers identify important visual locations for final predictions, aiding in the understanding of visual hallucination. Our method demonstrates faster and more effective results compared to existing interpretability approaches. Code: \url{https://github.com/zepingyu0512/llava-mechanism}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。