arXiv:2608.08536cs.LG2026-08

用图神经网络学习变压器中的计算电路,实现跨任务的高效定位。

Can Graph Learning Learn Circuits?

  • 将计算路径建模为图边,用GNN学习跨模型-任务的电路结构。
  • 在16个测试案例上达到0.902的中位边AUROC,接近现有最佳方法。
  • 首次尝试用图学习框架统一解决电路定位问题,适合可解释性研究者。

电路定位是一项机制可解释性任务,目标是识别Transformer计算图中足以重现特定行为的稀疏子图。现有方法通常对每个模型-任务组合独立进行定位。本文将电路定位视为图机器学习问题,其中计算图的边代表计算路径,图神经网络(GNN)建模这些路径间的交互。我们提出图电路学习(GCL),一种监督式、可复用的框架,在多个模型-任务对上训练GNN,并应用于未见情况。为提供足够数据,我们在InterpBench基准上新增了来自TracrBench程序的案例。在14种评估配置中,最优方案在16个原始保留的InterpBench案例上达到中位边AUROC 0.902(四分位区间[0.861, 0.942]),接近现有公布的InterpBench中位数0.910(EAP-IG),但仍低于ACDC的0.959。若移除所有消息传递边,中位数降至0.825。我们还适配了PGExplainer这一GNN可解释性方法用于电路定位,在相同案例上获得0.858的中位边AUROC。初步结果表明,图机器学习为电路定位提供了自然且有潜力的新视角,期望推动两个领域更紧密交流。

原文摘要 · Abstract (English)

Circuit localization is a mechanistic interpretability task whose goal is to identify a sparse subgraph of a transformer's computation graph sufficient to reproduce a particular behavior. Most established methods localize circuits independently for each model--task pair. We instead frame circuit localization as a graph machine learning problem in which the edges of a computation graph represent computational pathways, and graph neural networks (GNNs) model interactions among these pathways. We introduce Graph Circuit Learning (GCL), a supervised, amortized framework that trains a GNN across multiple model--task pairs and applies it to unseen cases. To provide sufficient data, we augment the InterpBench benchmark with additional cases derived from the TracrBench programs. Of the 14 evaluated GCL configurations, the highest scored a median edge AUROC of $0.902$ (interquartile interval $[0.861, 0.942]$) on the 16 original held-out InterpBench cases. This is close to the published InterpBench median of $0.910$ for EAP-IG while remaining below ACDC's $0.959$. Removing all message-passing edges reduces the median to $0.825$. We also adapt PGExplainer, a GNN explainability method, to circuit localization, obtaining a median edge AUROC of $0.858$ on the same cases. These preliminary results suggest that graph machine learning offers a natural and potentially powerful perspective on circuit localization, and we hope this perspective encourages closer exchange between the two communities.

图神经网络可解释性电路定位

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。