揭示GPT-2中注意力陷阱的根源，为缓解提供新思路。
A Mechanistic Account of Attention Sinks in GPT-2: One Circuit, Broader Implications for Mitigation
- 发现注意力集中在首位置源于查询偏置、位置编码变换与键投影结构的交互。
- 三要素任一缺失模型仍出现注意力陷阱，说明其成因具有多样性。
- 适合关注模型可解释性与注意力机制优化的研究者阅读。
Transformer模型普遍存在注意力陷阱：对首个位置赋予过高关注度。本文研究基于学习查询偏置和绝对位置嵌入的GPT-2类模型，结合结构分析与因果干预，在自然语言、数学和代码输入上验证发现，该现象源于（i）学习到的查询偏置、（ii）第一层MLP对位置编码的变换，以及（iii）键投影中的结构特性。关键的是，这三个组件各自均可被移除，且模型在去除任一后仍表现出注意力陷阱，表明不同架构中可能存在多种独立的成因路径。这一发现为缓解注意力陷阱提供了方向，并推动对陷阱产生原因的更广泛探究。
原文摘要 · Abstract (English)
Transformers commonly exhibit an attention sink: disproportionately high attention to the first position. We study this behavior in GPT-2-style models with learned query biases and absolute positional embeddings. Combining structural analysis with causal interventions, validated across natural-language, mathematical, and code inputs, we find that the sink arises from the interaction among (i) a learned query bias, (ii) the first-layer MLP transformation of the positional encoding, and (iii) structure in the key projection. Crucially, each component we identify is individually dispensable: architectures omitting each of them robustly exhibit sinks. This indicates that attention sinks may arise through distinct circuits across architectures. These findings inform mitigation of sinks, and motivate broader investigation into why sinks emerge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。