通过定向探针追踪机制，揭示模型内部概念形成的真正原因。
How Do Linear Probes Emerge? A Circuit-Tracing Framework with Concept-Targeted Attribution
- 用线性探针方向训练归因图，定位特定概念的内部计算路径。
- 图级特征能准确预测探针性能（ρ=0.91, R²=0.84），局部特征识别关键组件。
- 可区分探针相关与输出相关的机制，适用于安全关键概念审计。
传统的跨层译码归因图通常用于解释模型为何对某个下一个词赋予高概率。本文提出概念定向归因（CTA），其以线性探针方向为训练目标，生成针对特定探针的归因图。由此得到的探针特异性电路可解释提示中内部概念表征的产生原因，而无需依赖生成结果是否体现该概念。利用跨层译码器，我们发现这些探针定向图具有可预测结构：图级特征在四个广泛研究的概念类别上能准确预测探针精度（ρ=0.91, R²=0.84），局部特征则识别出每条提示中的稀疏驱动成分。该结果将探针表现与可解释的电路结构联系起来，使我们不仅能判断探针是否有效，还能明确其有效的内部计算机制。因果消融实验进一步表明，探针相关与词元目标归因图捕捉的是功能上不同的机制：移除探针相关特征会降低内部概念得分，但几乎不影响生成词；而移除词元相关特征则在92%至100%情况下改变生成词，对探针得分影响极小。CTA提供了一个从行为准确性迈向机制解释的框架，支持对内部概念表征进行更精细的审计，包括安全敏感场景。代码已开源于 https://github.com/vedantpalit/concept-targeted-attribution。
原文摘要 · Abstract (English)
Transcoder attribution graphs are usually trained to explain why a model assigns high probability to a particular next token. We introduce Concept-Targeted Attribution (CTA), which instead trains attribution graphs with respect to a linear probe direction. CTA therefore yields probe-specific circuits that explain why an internal concept representation arises in a prompt, independently of whether it is expressed in the generated token. Using Cross-Layer Transcoders, we show that these probe-targeted graphs contain predictive structure: graph-level features predict probe accuracy across four widely studied concept categories ($ρ= 0.91$, $R^2 = 0.84$), while local features identify the sparse components driving per-prompt classification. This connects probe performance to interpretable circuit structure, allowing us to ask not only whether a probe works, but which internal computations make it work. Causal ablations further show that probe-targeted and logit-targeted graphs capture functionally distinct mechanisms. Removing probe-relevant features reduces internal concept scores while largely preserving generated tokens, whereas removing logit-relevant features changes the generated token in 92% to 100% of cases with near-zero effect on probe scores. CTA provides a framework for moving from behavioral probe accuracy to mechanistic explanations of probe performance, enabling more detailed audits of internal concept representations, including safety-critical ones. Our code is available at https://github.com/vedantpalit/concept-targeted-attribution
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。