arXiv:2609.07876cs.CLcs.LG2026-09

发现大模型层间存在即时纠错机制,颠覆了残差流持续累积特征的传统认知。

LLM Layers Immediately Correct Each Other

论文配图:LLM Layers Immediately Correct Each Other
图 1 · 摘自论文原文
  • 相邻层通过系统性抵消彼此贡献,形成动态修正机制
  • 该机制在7个主流模型中出现5个,覆盖几乎所有文本类型
  • 解释了稀疏自编码器特征不明确、模型操控需强放大等现象

近期语言模型可解释性研究常使用稀疏自编码器(SAE)将残差流贡献分解为线性且语义明确的特征。这类方法通常被理解为识别出在残差流中持续存在并被后续层继承的特征。我们提出挑战:在5个主流开源模型家族中发现了变压器层纠错机制(TLCM),即相邻层系统性地抵消彼此的贡献。TLCM在预训练过程中产生,对上下文依赖性较强的词元作用最强,并根据前一层输出自适应调节纠正强度。通过层雅可比矩阵分析,我们发现其选择性修正特定子空间,同时强化其他子空间,这可用‘提出-拒绝’框架解释:各层提出候选特征,后续层选择性剔除不当项。这一动态表明,任意层的残差流包含瞬时提案与持久特征共存,有助于解释为何SAE特征描述特异性低、有效模型操控需极端特征放大,以及为何转换器相比SAE具有理论优势。

原文摘要 · Abstract (English)

Recent methods in language model interpretability employ techniques such as sparse autoencoders to decompose residual stream contributions into linear, semantically meaningful features. Such methods are commonly interpreted as identifying features that persist in the residual stream and that subsequent layers build upon. We challenge this view by identifying the Transformer Layer Correction Mechanism (TLCM), wherein adjacent transformer layers systematically counteract portions of each other's contributions. TLCM appears in 5 out of 7 major open-source model families and activates across nearly all tokens in diverse texts. We show that TLCM emerges during pretraining, operates most strongly on contextually dependent tokens, and adaptively calibrates its correction strength based on the preceding layer's output. Using the layer Jacobian, we further show that TLCM selectively corrects specific subspaces while reinforcing others, which we interpret through a ``propose-and-reject'' framework in which layers propose candidate features and subsequent layers selectively remove inappropriate ones. This dynamic suggests that the residual stream at any layer contains transient proposals alongside persistent features, helping explain why SAE feature descriptions often have low specificity, why effective model steering requires extreme feature amplification, and why transcoders hold a theoretical advantage over SAEs.

模型可解释性注意力机制残差流Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。