实验证明，Transformer在计数语言中依赖栈结构表示才能正确生成序列。
Causal Evidence of Stack Representations in Modeling Counter Languages Using Transformers
- 用线性探测提取模型隐藏状态中的栈深度方向
- 移除该方向后模型准确率降至0.3%，性能几乎崩溃
- 首次提供栈表示对模型性能的因果证据，适合关注机制解释的研究者
形式语言已被证明是理解Transformer内部机制的有效途径。以往研究显示，基于计数语言进行下一个词预测训练的Transformer会学习到与底层栈结构一致的表示。本文进一步探究这些表示的因果作用：通过线性探测从模型隐藏状态中预测每个标记的栈深度，并提取主代表方向。对该方向进行消融后，模型的序列生成准确率降至0.3%，表明栈表示不仅是被学习到的，更是模型性能所必需的因果成分。
原文摘要 · Abstract (English)
Formal languages have proven to be effective conduits to understand the inner mechanisms of transformers. Past work has shown that transformers trained on next token prediction over counter languages learn representations consistent with an underlying stack structure. Beyond representational analysis, this paper investigates the causal role of these representations. Linear probes are trained to predict the stack depth at each token from the model's hidden states, and a principal representation direction is extracted from the probe. Ablation of this direction from the model causes sequential accuracy to collapse to near 0%, providing strong empirical evidence that the stack representation is not just learned, but is causally necessary for model performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。