发现视觉语言模型的神经元存在特征混叠，导致组合推理失败。
Do VLMs Have Bad Eyes? Diagnosing Compositional Failures via Mechanistic Interpretability
- 通过可解释性分析发现CLIP视觉编码器的MLP层神经元同时表征多个特征。
- 特征混叠现象直接削弱了模型对物体属性组合的表示能力。
- 适合研究多模态模型机制、提升组合泛化能力的学者参考。
视觉语言模型（VLMs）在图像描述和视觉问答等任务中表现出色，但在组合泛化和对象绑定方面仍存在缺陷，限制其处理新物体与属性组合的能力。本文采用机制可解释性技术，探究这些失败的根本原因。研究发现，CLIP视觉编码器的MLP层中，单个神经元同时表征多个特征，这种“超叠加”现象直接阻碍了组合特征的准确表示，进而影响模型的组合推理与对象绑定能力。该研究为揭示VLMs组合失败的机制根源提供了初步线索。代码与补充结果详见 https://github.com/Mystic-Slice/Do-VLMs-Have-Bad-Eyes。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) have shown remarkable performance in integrating visual and textual information for tasks such as image captioning and visual question answering. However, these models struggle with compositional generalization and object binding, which limit their ability to handle novel combinations of objects and their attributes. Our work explores the root causes of these failures using mechanistic interpretability techniques. We show evidence that individual neurons in the MLP layers of CLIP's vision encoder represent multiple features, and this "superposition" directly hinders its compositional feature representation which consequently affects compositional reasoning and object binding capabilities. We hope this study will serve as an initial step toward uncovering the mechanistic roots of compositional failures in VLMs. The code and supporting results can be found https://github.com/Mystic-Slice/Do-VLMs-Have-Bad-Eyes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。