arXiv:2506.17052cs.LGcs.AI2025-06被引 4

提出可泛化注意力模块发现方法,实现对复杂概念的精准定位与调控。

From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformers

  • 用向量相似度匹配概念与注意力头,实现无需先验知识的模块定位
  • 通过单标量调节使模型推理能力提升1.6%,安全防护降低72.7%
  • 方法不依赖特定领域,适用于语言与视觉模型

Transformer 在语言和视觉任务中取得了顶尖性能,推动对其内部机制的可解释性研究。现有归因方法多关注 MLP 神经元,且仅限于简单概念(如“巴黎位于法国”)。本文提出可扩展注意力模块发现(SAMD),将任意复杂概念表示为向量,通过余弦相似度匹配其对应注意力头,选取前K个得分最高的头构建概念关联模块。进一步提出标量注意力模块干预(SAMI),仅用一个标量参数即可增强或削弱概念影响。实验证明,模块位置在大模型后训练前后保持稳定,验证了多语言模型工作机制。利用SAMI,在HarmBench上实现72.7%的越狱成功率提升,GSM8K上推理能力提升1.6%。最后通过抑制视觉变换器在ImageNet上的分类准确率,证明该方法具有领域无关性。

原文摘要 · Abstract (English)

Transformers have achieved state-of-the-art performance across language and vision tasks. This success drives the imperative to interpret their internal mechanisms with the dual goals of enhancing performance and improving behavioral control. Attribution methods help advance interpretability by assigning model outputs associated with a target concept to specific model components. Current attribution research primarily studies multi-layer perceptron neurons and addresses relatively simple concepts such as factual associations (e.g., Paris is located in France). This focus tends to overlook the impact of the attention mechanism and lacks a unified approach for analyzing more complex concepts. To fill these gaps, we introduce Scalable Attention Module Discovery (SAMD), a concept-agnostic method for mapping arbitrary, complex concepts to specific attention heads of general transformer models. We accomplish this by representing each concept as a vector, calculating its cosine similarity with each attention head, and selecting the TopK-scoring heads to construct the concept-associated attention module. We then propose Scalar Attention Module Intervention (SAMI), a simple strategy to diminish or amplify the effects of a concept by adjusting the attention module using only a single scalar parameter. Empirically, we demonstrate SAMD on concepts of varying complexity, and visualize the locations of their corresponding modules. Our results demonstrate that module locations remain stable before and after LLM post-training, and confirm prior work on the mechanics of LLM multilingualism. Through SAMI, we facilitate jailbreaking on HarmBench (+72.7%) by diminishing "safety" and improve performance on the GSM8K benchmark (+1.6%) by amplifying "reasoning". Lastly, we highlight the domain-agnostic nature of our approach by suppressing the image classification accuracy of vision transformers on ImageNet.

注意力机制可解释性模型干预

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。