解析视觉语言模型中神经元的功能分工，提升AI可解释性。
Deciphering Functions of Neurons in Vision-Language Models
- 通过分析输入图像和文本激活情况，区分视觉、文本和跨模态神经元。
- 在LLaVA模型上发现三类神经元占比分别为38%、29%和33%。
- 用GPT-4o自动解释神经元功能，并设计模拟器验证视觉神经元解释可靠性。
开源视觉语言模型（VLMs）的快速发展推动了多领域应用，但模型透明性与可解释性对构建可信可靠的AI系统至关重要。本研究深入探索VLM内部机制，旨在解析单个神经元的功能。我们观察神经元对视觉标记和文本标记的激活响应，发现存在仅响应视觉信息、仅响应文本信息或同时响应两者的神经元，分别称为视觉神经元、文本神经元和多模态神经元。为此，我们构建了一个基于GPT-4o辅助的自动化神经元解释框架。针对视觉神经元，提出一种激活模拟器以评估解释的可靠性。在代表性模型LLaVA上进行系统统计分析，揭示了三类神经元的行为特征与分布规律。
原文摘要 · Abstract (English)
The burgeoning growth of open-sourced vision-language models (VLMs) has catalyzed a plethora of applications across diverse domains. Ensuring the transparency and interpretability of these models is critical for fostering trustworthy and responsible AI systems. In this study, our objective is to delve into the internals of VLMs to interpret the functions of individual neurons. We observe the activations of neurons with respects to the input visual tokens and text tokens, and reveal some interesting findings. Particularly, we found that there are neurons responsible for only visual or text information, or both, respectively, which we refer to them as visual neurons, text neurons, and multi-modal neurons, respectively. We build a framework that automates the explanation of neurons with the assistant of GPT-4o. Meanwhile, for visual neurons, we propose an activation simulator to assess the reliability of the explanations for visual neurons. System statistical analyses on top of one representative VLM of LLaVA, uncover the behaviors/characteristics of different categories of neurons.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。