让视觉语言模型的推理过程可解释，帮人发现它出错的真实原因。
VisualScratchpad: Inference-time Visual Concepts Analysis in Vision Language Models
- 用稀疏自编码器提取视觉概念，关联到文本词元分析跨模态使用情况。
- 发现模型三大隐藏错误：对齐不足、误导性视觉特征、未利用隐含线索。
- 适合想调试模型、理解错误根源的研究者和工程师使用。
高性能视觉语言模型仍会给出错误答案，但其失败模式往往难以解释。为使模型内部更透明并支持系统性调试，我们提出 VisualScratchpad，一个在推理时进行视觉概念分析的交互式界面。通过在视觉编码器上应用稀疏自编码器，并利用文本到图像注意力将生成的视觉概念与文本词元关联，可检查哪些视觉概念被视觉编码器捕获且被语言模型使用。VisualScratchpad 还提供词元-隐状态热力图，提示有效概念消融所需的最小隐状态集合，用于因果分析。案例研究揭示了三个未充分探索的失败模式：跨模态对齐有限、误导性视觉概念、未使用的隐藏线索。项目页面：https://hyesulim.github.io/visual_scratchpad_projectpage/
原文摘要 · Abstract (English)
High-performing vision language models still produce incorrect answers, yet their failure modes are often difficult to explain. To make model internals more accessible and enable systematic debugging, we introduce VisualScratchpad, an interactive interface for visual concept analysis during inference. We apply sparse autoencoders to the vision encoder and link the resulting visual concepts to text tokens via text-to-image attention, allowing us to examine which visual concepts are both captured by the vision encoder and utilized by the language model. VisualScratchpad also provides a token-latent heatmap view that suggests a sufficient set of latents for effective concept ablation in causal analysis. Through case studies, we reveal three underexplored failure modes: limited cross-modal alignment, misleading visual concepts, and unused hidden cues. Project page: https://hyesulim.github.io/visual_scratchpad_projectpage/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。