通过稀疏分解揭示象棋模型的深层推理机制
Tracing the Thought of a Grandmaster-level Chess-Playing Transformer
- 用稀疏替换层分解MLP和注意力模块，还原核心计算路径
- 发现可验证的战术思考模式，证明模型存在并行推理行为
- 为超人级系统的决策机制提供可解释性分析框架
尽管现代Transformer神经网络在国际象棋等推理任务中达到大师级水平，其内部计算过程仍高度不透明。针对Leela Chess Zero（LC0），本文提出一种稀疏分解框架，通过稀疏替换层对MLP和注意力模块进行分解，捕获了LC0的主要计算过程。详细案例研究表明，这些路径揭示出丰富且可解释的战术考量，且可实证验证。进一步引入三项量化指标，表明LC0的并行推理行为与策略头架构的归纳偏置一致。据我们所知，这是首个同时在MLP和注意力模块上对Transformer内部计算进行可解释性分解的工作。结合稀疏替换层与因果干预，全面理解了高级战术推理，为超人系统提供了关键机制洞察。代码已开源：https://github.com/JacklE0niden/Leela-SAEs。
原文摘要 · Abstract (English)
While modern transformer neural networks achieve grandmaster-level performance in chess and other reasoning tasks, their internal computation process remains largely opaque. Focusing on Leela Chess Zero (LC0), we introduce a sparse decomposition framework to interpret its internal computation by decomposing its MLP and attention modules with sparse replacement layers, which capture the primary computation process of LC0. We conduct a detailed case study showing that these pathways expose rich, interpretable tactical considerations that are empirically verifiable. We further introduce three quantitative metrics and show that LC0 exhibits parallel reasoning behavior consistent with the inductive bias of its policy head architecture. To the best of our knowledge, this is the first work to decompose the internal computation of a transformer on both MLP and attention modules for interpretability. Combining sparse replacement layers and causal interventions in LC0 provides a comprehensive understanding of advanced tactical reasoning, offering critical insights into the underlying mechanisms of superhuman systems. Our code is available at https://github.com/JacklE0niden/Leela-SAEs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。