发现方向和大小在变压器中分别影响注意力与语法判断。
Disentangling Direction and Magnitude in Transformer Representations: A Double Dissociation Through L2-Matched Perturbation Analysis
- 用等长扰动分析分离方向与大小的影响
- 方向扰动使语言建模损失增42.9，大小扰动使语法准确率降20.4%
- 适合关注模型可解释性与编辑的研究者
Transformer隐藏状态以高维向量编码信息，但方向（表示空间中的朝向）和大小（向量范数）是否承担不同功能尚不明确。通过对Pythia系列模型的研究，我们发现显著的交叉解耦现象：角度扰动对语言建模损失造成的损害比大小扰动高出42.9，而大小扰动对句法处理的损害则远超方向扰动（在主谓一致任务上准确率下降20.4%对比1.6%）。这一发现得益于L2匹配扰动分析方法，确保角度与大小扰动产生相同的欧几里得位移。因果干预表明，角度损伤主要通过注意力路径传播（注意力修复可恢复28.4%损失），而大小损伤部分通过层归一化路径传播（层归一化修复可恢复29.9%损失）。该模式在Pythia架构内部多个规模下均复现。结果表明，在基于层归一化的架构中，方向与大小支持部分独立的计算角色：方向偏好影响注意力路由，大小则调节精细句法判断的处理强度。在RMSNorm架构中观察到不同模式，表明这种解耦依赖于具体架构设计。研究结果修正了线性表示假设，并对模型编辑与可解释性研究具有启示。
原文摘要 · Abstract (English)
Transformer hidden states encode information as high-dimensional vectors, yet whether direction (orientation in representational space) and magnitude (vector norm) serve distinct functional roles remains unclear. Studying Pythia-family models, we discover a striking cross-over dissociation: angular perturbations cause up to 42.9 more damage to language modeling loss, while magnitude perturbations cause disproportionately more damage to syntactic processing (20.4% vs.1.6% accuracy drop on subject-verb agreement).This finding is enabled by L2-matched perturbation analysis, a methodology ensuring that an gular and magnitude perturbations achieve identical Euclidean displacements. Causal intervention reveals that angular damage flows substantially through the attention pathways (28.4% loss recovery via attention repair), while magnitude damage flows partly through the LayerNorm pathways(29.9% recovery via LayerNorm repair). These patterns replicate across scales within the Pythia architecture family. These findings provide evidence that direction and magnitude support partially distinct computational roles in LayerNorm based architectures. The direction preferentially affects attentional routing, while magnitude modulates processing intensity for fine-grained syntactic judgments. We find different patterns in RMSNorm-based architectures, suggesting that the dissociation depends on architectural choices. Our results refine the linear representation hypothesis and have implications for model editing and interpretability research
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。