用合成模型验证Transformer内部状态是否具有马尔可夫转移结构
Markovian Circuit Tracing for Transformer State Dynamic
- 设计合成隐马尔可夫任务，精确控制状态与转移
- 小模型近乎达到贝叶斯最优预测，误差仅0.0138
- 状态强制实验表明激活值能恢复粗粒度转移信号
许多序列计算更适合通过内部状态演化来理解，而非孤立的局部电路。本文提出马尔可夫电路追踪（MCT），一种用于检验Transformer激活是否包含粗粒度状态转移结构的诊断流程。基准测试采用合成隐马尔可夫模型（HMM）任务，其中潜在状态、转移矩阵、贝叶斯信念向量、贝叶斯最优预测及强制状态反事实目标均精确已知。在六个HMM家族、每个家族三个随机种子下，微小因果Transformer学习到接近贝叶斯最优的下一个词预测器，平均超出贝叶斯损失仅为0.0138。残差激活中包含部分贝叶斯信念信息。从这些激活中提取的状态抽象能恢复粗粒度转移信号，最显著出现在持久态和低状态情形，而在模糊观测和六状态情形较弱。最清晰的结果来自状态强制：将恢复的状态中心点代入，模型与精确反事实目标的KL散度从0.1957降至0.0532，优于错误状态、平均激活、随机激活和洗牌标签等对照组。贡献在于提供一个可控的基准和评估框架，用于Transformer状态动态的可解释性研究，其中MCT作为简单参考流程。
原文摘要 · Abstract (English)
Many sequence computations are easier to study as movement through internal states than as isolated local circuits. We introduce Markovian Circuit Tracing (MCT), a diagnostic pipeline for testing whether transformer activations contain coarse state-transition structure. The benchmark uses synthetic Hidden Markov Model (HMM) tasks where latent states, transition matrices, Bayesian belief vectors, Bayes-optimal predictions, and forced-state counterfactual targets are known exactly. Across six HMM families and three seeds per family, tiny causal transformers learn near-Bayes next-token predictors, with mean excess loss over Bayes of 0.0138. Residual activations contain partial Bayesian belief information in this controlled synthetic benchmark. State abstractions extracted from these activations recover coarse transition signal, strongest in persistent and lower-state regimes, and weaker in ambiguous-emission and six-state regimes. The clearest result comes from state forcing. Patching a recovered-state centroid reduces KL to the exact HMM counterfactual target from 0.1957 in the unpatched model to 0.0532 on average, beating wrong-state, mean-activation, random-activation, and shuffled-label controls. The contribution is a controlled benchmark and evaluation framework for transformer state-dynamics interpretability, with MCT as a simple reference pipeline
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。