arXiv:2607.18921cs.LGcs.AI2026-07被引 1

同一模型不同提取方式会得出不同电路结论,影响对机制的理解。

Circuit Claims Depend on What Is Extracted and How It Is Compared

论文配图:Circuit Claims Depend on What Is Extracted and How It Is Compared
图 1 · 摘自论文原文
  • 通过改变提取对象和比较方式,同一模型可生成差异巨大的电路
  • 组件重叠率低且受提取方法影响,部分情况趋近随机水平
  • 关注注意力头集合与电路规模排名更稳定,适合做结论对比

电路提取旨在识别保留目标行为的一小部分模型组件,其结果常被视为该行为的机制。我们指出这种解读是不确定的:仅保持行为无法唯一确定一个电路,因为结论依赖于报告哪个电路以及如何比较两个电路。我们在一个合成的Lean定理证明预测基准中具体展示了这一点——通过固定证明规则但随机化表面形式,使提取出的电路差异可归因于选择而非任务本身。在相同Transformer模型的不同密集与权重稀疏检查点(多数权重被约束为零)上,评估原子(单规则)和组合(多规则)证明时,我们改变了报告的提取对象(紧凑的预测保留电路、包含周边读写与路由结构的广义图、或满足后消融损失阈值的最小子图),以及是否将注意力头的查询与键联合表示。组件间边重合率低且对选择敏感,有时降至随机基线水平;而两种粗粒度摘要则保持稳定:所选注意力头集合,以及在不同监督检查点初始化强化学习(RL)时条件的电路规模排序。组合证明上RL带来的最大精度提升伴随着最丰富的额外结构。因此,只有明确报告的电路、提取所用的剪枝阈值以及比较层级,电路层面的结论才具有明确定义。我们据此提炼出电路提取研究的报告规范。

原文摘要 · Abstract (English)

Circuit extraction identifies a small set of model components whose presence preserves a target behavior under ablation, and the resulting circuit is often read as the mechanism behind that behavior. We argue that this reading is under-determined: preserving behavior does not single out one circuit, because the claim it supports depends on which circuit is reported and how two circuits are compared. We make this concrete in a synthetic Lean tactic-prediction benchmark -- predicting the next step of a proof -- where fixed proof rules with randomized surface form let differences between extracted circuits be attributed to these choices rather than to the task. Across dense and weight-sparse checkpoints (most weights constrained to zero) of the same transformer, evaluated on atomic (single-rule) and compositional (multi-rule) proofs, we vary which extracted object is reported (a compact prediction-preserving circuit, a broader graph that also keeps surrounding read, write, and routing structure, or the smallest subgraph meeting a post-ablation loss threshold), and whether each attention head's query and key are represented jointly or separately. Exact component-to-component edge overlap is low and sensitive to these choices, at times dropping to a random baseline, while two coarser summaries stay stable: the set of selected attention heads, and the circuit-size ranking of conditions that differ in which supervised checkpoint initializes reinforcement learning (RL). The largest accuracy gains from RL on compositional proofs come with the most structure beyond the atomic circuits. A circuit-level claim is therefore well defined only once one states which circuit is reported, the pruning threshold used to extract it, and the level at which circuits are compared. We distill these requirements into a reporting practice for circuit-extraction studies.

神经网络解释电路提取机制分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。