Transformer解数独时构建了基于约束结构的内部世界模型。
Transformers Linearly Represent Highly Structured World Models
- 用8层Transformer训练解数独，分析其内部计算机制
- 发现模型以行、列、宫为单位组织信息，而非逐格存储
- 找到专门检测唯一可能数字的神经元电路，可解释且稀疏
当Transformer在序列推理轨迹上训练时，是否会构建底层任务的内部模型？若是，这些内部表示的结构是否反映领域本身的结构？我们对一个8层Transformer在数独求解轨迹上的训练进行了机制性分析，得出两个结论：第一，模型构建了一个子结构世界模型——它并未像人类分析师预期的那样逐单元表示棋盘状态，而是围绕数独约束所作用的行、列和宫来组织信息；第二，我们识别出一个裸单数电路：位于最后一层MLP中的一小部分专用神经元，每个神经元独立检测特定单元格中唯一可能的数字，并可靠地推动该数字的确定。这些发现表明，涌现的世界模型几何结构由领域约束代数决定，而非表面呈现形式；且由此产生的决策电路具有稀疏性、单义性和完全可解释性。更广泛而言，这证明机制可解释性工具能够还原Transformer解决组合推理任务的端到端算法过程。
原文摘要 · Abstract (English)
Do transformers, when trained on sequential reasoning traces, build internal models of the underlying task? And if so, does the structure of those internal representations mirror the structure of the domain? We train an 8-layer transformer on Sudoku solving traces and perform a mechanistic analysis of its internal computation. We establish two results. First, the model builds a substructure world model: it does not represent the board state cell by cell, as a human analyst would expect, but organizes information around the rows, columns, and boxes that Sudoku's constraints act on. Second, we identify a naked-single circuit: a small set of dedicated neurons in the final MLP layer, each individually detecting when exactly one digit remains possible for a specific cell, and reliably promoting that digit. These findings show that the geometry of an emergent world model is shaped by the constraint algebra of the domain, not its surface presentation, and that the resulting decision circuit is sparse, monosemantic, and fully interpretable. More broadly, they demonstrate that mechanistic interpretability tools can recover an end-to-end algorithmic account of how a transformer solves a combinatorial reasoning task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。