发现多模态大模型中关键感知神经元，仅关闭2%就让性能暴跌。
MINER: Mining the Underlying Pattern of Modality-Specific Neurons in Multimodal Large Language Models
- 通过四阶段流程挖掘多模态模型中的特定模态神经元。
- 关闭仅2%的特定神经元，模型性能从0.69降至0.31。
- 揭示了不同模态信息在低层汇聚及最后令牌的融合机制。
近年来，多模态大语言模型（MLLMs）快速发展，集成多种模态应用于多样化场景。然而，可解释性不足仍是其在需决策透明的场景中应用的主要障碍。现有神经元层面的解释方法多聚焦知识定位或语言与领域特异性分析，对多模态探索仍显不足。为此，我们提出MINER，一种可迁移的框架，用于挖掘MLLMs中的模态特定神经元（MSNs），包含四个阶段：(1) 模态分离，(2) 重要性评分计算，(3) 重要性评分聚合，(4) 模态特定神经元选择。在六个基准和两种代表性MLLMs上的大量实验表明：(I) 仅禁用2%的MSNs即显著降低模型性能（Qwen2-VL从0.56降至0.24，Qwen2-Audio从0.69降至0.31）；(II) 不同模态主要在低层汇聚；(III) MSNs影响各模态关键信息向最后一个标记的融合方式；(IV) 发现两个值得深入研究的现象，即语义探针与语义端粒。代码已开源。
原文摘要 · Abstract (English)
In recent years, multimodal large language models (MLLMs) have significantly advanced, integrating more modalities into diverse applications. However, the lack of explainability remains a major barrier to their use in scenarios requiring decision transparency. Current neuron-level explanation paradigms mainly focus on knowledge localization or language- and domain-specific analyses, leaving the exploration of multimodality largely unaddressed. To tackle these challenges, we propose MINER, a transferable framework for mining modality-specific neurons (MSNs) in MLLMs, which comprises four stages: (1) modality separation, (2) importance score calculation, (3) importance score aggregation, (4) modality-specific neuron selection. Extensive experiments across six benchmarks and two representative MLLMs show that (I) deactivating ONLY 2% of MSNs significantly reduces MLLMs performance (0.56 to 0.24 for Qwen2-VL, 0.69 to 0.31 for Qwen2-Audio), (II) different modalities mainly converge in the lower layers, (III) MSNs influence how key information from various modalities converges to the last token, (IV) two intriguing phenomena worth further investigation, i.e., semantic probing and semantic telomeres. The source code is available at this URL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。