arXiv:2605.08348cs.CL2026-05被引 3

发现语言模型电路缺乏任务特异性,共享组件影响多任务表现

How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

论文配图:How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits
图 1 · 摘自论文原文
  • 通过边缘归因修补法分析六任务七模型的电路复用情况
  • 同一任务内组件复用率高,删除后准确率下降超100%相对值
  • 跨任务电路重叠大,多数组件非任务专属,影响干预精准性

机制可解释性中的电路框架旨在识别对模型行为有因果影响的稀疏子图,通常通过必要性和充分性评估。本文测量了任务内电路复用率,并研究两个较少关注的性质:一致性(组件在任务内的重复出现)和特异性(组件仅属于某一任务)。基于六项任务和七种模型的边缘归因修补实验,发现任务内组件复用率高,且这些共享组件对任务性能至关重要,其移除可导致高达约100%的相对准确率下降。然而,电路并非任务特异:移除某一任务的电路,对其他任务性能的损害程度与自身任务相当。这源于不同任务间电路存在显著重叠,且这些重叠部分对性能具有因果重要性。尽管少数组件具有任务特异性,但其贡献仅占电路性能的一小部分。整体表明,虽注意力头和MLP层级别的电路能识别关键组件,但其缺乏任务特异性,挑战了电路在精准理解与干预模型行为上的有效性。

原文摘要 · Abstract (English)

The circuits framework in mechanistic interpretability aims to identify causally important sparse subgraphs of model components, typically evaluated by measuring necessity and sufficiency. We measure circuit reuse, the proportion of components shared across per-example circuits within a task, and investigate two less-studied properties of this: consistency, the recurrence of components within a task, and specificity, their uniqueness to a task. Using edge attribution patching across six tasks and seven models, we find that within-task reuse is high and that shared components are necessary for task performance, with ablations causing up to $\sim$100% relative accuracy drops. However, circuits turn out not to be task-specific: ablating one task's circuit damages another task's performance about as much as that task's own circuit does. We discover that this is due to substantial overlap between circuits across tasks, which are causally important for performance. Some circuits do contain a smaller set of task-specific components, but these account for only a modest portion of circuit performance. Overall, our findings suggest that while circuit discovery at the level of attention heads and MLP layers identifies important components, their lack of task-specificity raises questions about the degree to which circuits can support targeted understanding and intervention on model behavior.

机制可解释电路分析模型行为任务特异性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。