通过参数推断注意力头功能，无需推理或训练。
Inferring Functionality of Attention Heads from their Parameters
- 从参数直接推断注意力头的功能，不依赖运行时行为。
- 在6个主流模型上验证，与实际输出高度相关且影响预测结果。
- 发现被忽略的隐藏功能，揭示模型架构的通用性与偏差。
注意力头是大语言模型的核心组件。以往研究多聚焦于特定任务或回路中的推理行为分析。本文提出MAPS（Mapping Attention head ParameterS）框架，无需模型训练或推理，仅通过参数即可推断注意力头的功能。我们在6个主流大语言模型中评估了20种预定义操作，结果表明：MAPS的推断与推理时的输出高度相关，且对模型预测具有因果影响。该方法揭示了此前研究未关注的注意力头功能，并提供了关于功能通用性与模型架构偏见的深入洞察。此外，我们构建了自动化流程，可为任意注意力头生成合理功能描述，经人工评估多数结果可信，且展现出功能多样性。
原文摘要 · Abstract (English)
Attention heads are one of the building blocks of large language models (LLMs). Prior work on investigating their operation mostly focused on analyzing their behavior during inference for specific circuits or tasks. In this work, we seek a comprehensive mapping of the operations they implement in a model. We propose MAPS (Mapping Attention head ParameterS), an efficient framework that infers the functionality of attention heads from their parameters, without any model training or inference. We showcase the utility of MAPS for answering two types of questions: (a) given a predefined operation, mapping how strongly heads across the model implement it, and (b) given an attention head, inferring its salient functionality. Evaluating MAPS on 20 operations across 6 popular LLMs shows its estimations correlate with the head's outputs during inference and are causally linked to the model's predictions. Moreover, its mappings reveal attention heads of certain operations that were overlooked in previous studies, and valuable insights on function universality and architecture biases in LLMs. Next, we present an automatic pipeline and analysis that leverage MAPS to characterize the salient operations of a given head. Our pipeline produces plausible operation descriptions for most heads, as assessed by human judgment, while revealing diverse operations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。