发现大模型语言核心仅依赖少数神经元，删掉几个就崩溃。
Sparse Neuron Ablation Triggers Catastrophic Collapse of the Language Core in Large Vision-Language Models
- 通过逐步删除神经元，定位关键脆弱点。
- 删4个神经元即可让LLaVA模型彻底失效。
- 漏洞集中在语言模块的下投影层，适合研究模型安全者看。
大型视觉语言模型（LVLMs）展现出强大的多模态理解能力，但从机制可解释性角度，其功能维持结构仍不清晰。本文提出一致激活神经元（CAN）方法，通过渐进式神经元删减，识别出引发灾难性崩溃的关键神经元，并用于分析代表性7B规模LVLMs的结构性脆弱性。实验表明，仅删除4个神经元即可触发 exttt{LLaVA-1.5-7b-hf} 的崩溃，而 exttt{InstructBLIP-vicuna-7b} 也仅需数千个神经元即引发整体失效，均仅为模型参数的小部分。值得注意的是，这些关键神经元主要集中于语言模型部分，特别是其下投影层，而非视觉组件。此外观察到稳定的两阶段崩溃模式：先表达能力退化，后突然完全失效。结果揭示，LVLM的功能依赖于语言主干中一个稀疏的神经元子集，为模型功能结构与脆弱性提供了机制层面的洞察。
原文摘要 · Abstract (English)
Large Vision-Language Models (LVLMs) have shown impressive multimodal understanding capabilities, yet the structures that sustain their functionality remain poorly understood from a mechanistic interpretability standpoint. We propose Consistently Activated Neurons (CAN), a progressive neuron ablation method to identify critical neurons whose removal triggers catastrophic collapse, and use it to investigate structural vulnerabilities in representative 7B LVLMs. Experiments reveal that catastrophic collapse can be triggered by ablating as few as four neurons in \texttt{LLaVA-1.5-7b-hf} and a few thousand in \texttt{InstructBLIP-vicuna-7b}, both representing a small fraction of model parameters. Notably, critical neurons are predominantly localized in the language model, particularly in its down-projection layer, rather than in the vision components. We also observe a consistent two-stage collapse pattern: initial expressive degradation followed by sudden, complete collapse. These findings reveal that LVLM functionality depends on a sparse subset of neurons concentrated in the language backbone, offering mechanistic insights into how their functionality is structured and where these models are most vulnerable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。