发现大模型能力由少数注意力头支撑，可高效定位并干预。
Compressed Sensing for Capability Localization in Large Language Models
- 基于压缩感知思想，通过少量删头实验定位关键注意力头。
- 仅删5个特定头，核心能力下降60%但其他任务基本不受影响。
- 适用于模型可解释性、编辑与安全研究，支持多模型验证。
大型语言模型具备数学推理、代码生成和语言行为等多种能力。我们发现Transformer架构中存在少量对特定能力至关重要的注意力头。仅移除5个任务相关头，即可使标准评测中该能力性能下降高达60%，而对无关任务影响甚微。本文提出一种基于压缩感知的方法，利用这些头的稀疏性，通过策略性删除和少量模型评估即可识别关键组件。在1B至14B参数的Llama与Qwen模型上，针对数学能力和代码生成等多样化能力进行验证，揭示出专用能力依赖于稀疏且功能独立的模块化结构。结果表明，能力定位是Transformer语言模型的一般组织原则，对可解释性、模型编辑与人工智能安全具有重要意义。代码已开源：https://github.com/locuslab/llm-components。
原文摘要 · Abstract (English)
Large language models (LLMs) exhibit a wide range of capabilities, including mathematical reasoning, code generation, and linguistic behaviors. We show that Transformer architectures contain small subsets of attention heads that are necessary for certain capabilities. Zeroing out as few as five task-specific heads can degrade performance by up to $60\%$ on standard benchmarks measuring the capability of interest, while largely preserving performance on unrelated tasks. We introduce a compressed sensing-based method that exploits the sparsity of these heads to identify them via strategic knockouts and a small number of model evaluations. We validate these findings across Llama and Qwen models ranging from 1B to 14B parameters and a diverse set of capabilities including mathematical abilities and code generation, revealing a modular organization in which specialized capabilities are dependent on sparse, functionally distinct components. Overall, our results suggest that capability localization is a general organizational principle of Transformer language models, with implications for interpretability, model editing, and AI safety. Code is released at https://github.com/locuslab/llm-components.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。