让模型行为集中在更小的因果电路中,便于理解与验证。
Circuit Condensation: Post-Training that Concentrates a Behavior's Causal Circuit

- 通过修剪低贡献边并训练低秩适配器,逐步压缩因果电路。
- 平均缩小8.1倍,最多达316倍,且任务性能不受影响。
- 适合想深入理解模型内部机制的研究者和开发者。
机制可解释性常通过电路——承载行为的组件与连接——来解释。冻结发现通常返回数百条边,难以逐一检查、比较或验证。我们提出电路浓缩(Circuit Condensation),通过后训练将行为集中到更小的因果图中。每轮修剪低重要性边,并训练低秩适配器以保留原始行为,仅当任务性能与泛化能力保持时才保留剪枝结果。在四个行为和八个模型上,30/32设置下浓缩电路比最强冻结基线更小,平均缩小8.1倍,最多达316倍。重复搜索但不更新权重时,32组中有29组得到更大电路,表明权重更新而非搜索本身驱动压缩。测试19个电路的全部子集发现,11个不可再缩减,其余存在可移除边。成对消融揭示边间依赖,说明其效应不能独立理解。在间接宾语识别任务中,浓缩方法隔离出24个注意力头,其中17个有文献支持作用,而对应冻结电路含61个头、36个未记录作用:这是已知机制的充分子电路,而非重建。最终电路能追踪原模型的下一个词分布,并预测其错误。
原文摘要 · Abstract (English)
One approach to mechanistic interpretability explains behavior through circuits: the components and connections that carry it. Frozen discovery often returns hundreds of edges, making them hard to inspect, compare, or verify exhaustively. We introduce Circuit Condensation, which post-trains models to concentrate behaviors into smaller causal graphs. Each round prunes low-attribution edges and trains a low-rank adapter to match the original through what remains, retaining the cut only if task performance and general capability survive. Across four behaviors and eight models, condensed circuits are smaller than the strongest frozen baseline in 30 of 32 settings, by $8.1\times$ on average and up to $316\times$. Repeating the search without weight updates produces larger circuits in 29 of 32 settings, showing that weight updates, rather than search alone, drive the reduction. Testing every subset of 19 circuits finds 11 that cannot be reduced and reveals removable edges in the rest. Pair ablations expose dependencies between edges, showing that their effects cannot be understood independently. On indirect object identification, condensation isolates 24 heads, 17 of them with documented roles, against 61 heads and 36 undocumented ones for the matched frozen circuit: a sufficient sub-circuit of the published mechanism rather than a reconstruction of it. The resulting circuit tracks the original model's next-token distribution and predicts its errors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。