用栈结构模拟计数语言，发现Transformer自动学出栈式表示。
Emergent Stack Representations in Modeling Counter Languages Using Transformers
- 将计数语言转为栈结构，用深度模拟计数值
- 训练后模型在每个词元处显式编码栈深度
- 为理解Transformer的算法机制提供新视角
Transformer架构是现代语言模型的核心，但其内部工作机制仍不清晰。以往研究通过在形式语言上训练模型来探究其学习能力。本文扩展该方向，研究在4种计数语言上训练Transformer模型，并将这些语言等价地建模为栈结构，其中栈深对应计数变量值。通过探测每个输入词元的内部表示，发现模型作为下一个词元预测器训练后，能自发学习到类似栈的表示。这有助于揭示Transformer学习语言的算法细节，推动电路发现研究。
原文摘要 · Abstract (English)
Transformer architectures are the backbone of most modern language models, but understanding the inner workings of these models still largely remains an open problem. One way that research in the past has tackled this problem is by isolating the learning capabilities of these architectures by training them over well-understood classes of formal languages. We extend this literature by analyzing models trained over counter languages, which can be modeled using counter variables. We train transformer models on 4 counter languages, and equivalently formulate these languages using stacks, whose depths can be understood as the counter values. We then probe their internal representations for stack depths at each input token to show that these models when trained as next token predictors learn stack-like representations. This brings us closer to understanding the algorithmic details of how transformers learn languages and helps in circuit discovery.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。