用稀疏自编码器挖掘视觉语言模型中的多模态概念
Extraction and Analysis of Multimodal Concepts in Vision Language Models through Sparse Autoencoders
- 通过稀疏自编码器联合提取视觉、文本与多模态概念
- 在LLaVA-NeXT数据集上提升视觉概念质量45%
- 可系统识别融合双模态的深层概念，适合模型可解释性研究
视觉语言模型（VLMs）在图像描述和视觉问答（VQA）等需要图文联合理解的任务中表现优异，但其内部机制仍不清晰。近年来，稀疏自编码器（SAEs）成为解析VLM内部表征的有力工具，但现有方法多仅关注文本或视觉概念，忽视了跨模态概念，导致整合性概念被误判。此外，以往视觉方法生成的概念描述质量低，模糊不清，限制了对模型推理的理解。本文提出一种基于SAE的框架，用于从VLM中提取并分析视觉、文本及多模态概念。针对每个神经元，我们提出一个可被人类理解的概念候选，并使用余弦相似度计算该概念与数据样本的对齐程度。在LLaVA-NeXT VQA数据集上的实验表明，该框架相比现有SAE方法，视觉概念质量提升高达45%，同时保持高文本概念质量，并实现多模态概念的系统识别。本工作为理解VLM的语义空间提供了新视角，构建了区分三类概念的结构化方法。代码已开源：https://github.com/PHDLanza/Multidata_SAE
原文摘要 · Abstract (English)
Vision Language Models (VLMs) have demonstrated impressive performance in tasks requiring joint understanding of images and text, such as image captioning and Visual Question Answering (VQA), but our understanding of their internal processes remains limited. Recently, Sparse Autoencoders (SAEs) have emerged as a promising tool to support the interpretation of concepts encoded in VLMs. However, most SAE-based approaches focus only on textual or visual concepts separately, ignoring multimodal concepts. This limitation hinders a comprehensive understanding of VLMs, since concepts that integrate both modalities can be misclassified. Moreover, previous visual approaches often produce low-quality visual concept descriptions that are vague or incomplete, limiting their usefulness for understanding model reasoning. We propose a framework based on SAEs to extract and analyze visual, textual, and multimodal concepts from VLMs. For each neuron, we propose a candidate human-interpretable concept and compute the alignment between the concept and the dataset samples using cosine similarity scores. Experiments on a VQA dataset (LLaVA-NeXT) demonstrate that our framework improves visual concept quality by up to 45\% compared to existing SAE-based methods, while maintaining high textual concept quality and enabling systematic identification of multimodal concepts. This work contributes new insights into the conceptual space of VLMs, providing a structured approach to distinguish between visual, textual, and multimodal concepts. The code is available at https://github.com/PHDLanza/Multidata_SAE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。