提出可证明稳定的神经网络电路发现方法,提升解释可靠性。
Certified Circuits: Stability Guarantees for Mechanistic Circuits
- 用随机数据采样封装现有算法,确保电路组件不变性。
- 在多个任务上实现最高56%准确率提升,组件减少80%。
- 适合需要可信解释的模型调试与安全部署场景。
理解神经网络如何做出预测对调试、审计和部署至关重要。机制可解释性通过识别负责特定行为的最小子网络(即电路)来实现这一目标。然而,现有电路发现方法脆弱:其结果高度依赖概念数据集,且难以泛化到分布外数据,引发对其是否捕捉真实概念而非数据特异性伪影的质疑。本文提出「认证电路」(Certified Circuits),为电路发现提供可证明的稳定性保证。该框架通过随机数据子采样包装任意黑盒发现算法,确保电路组件(神经元或模型图中的边)的包含决策,在概念数据集的有界编辑距离扰动下保持不变。不稳定的组件被排除,从而获得更紧凑、更准确的电路。我们在三种架构(ResNet、ViT、GPT-2)上验证了该方法,涵盖视觉任务(ImageNet 及四个 OOD 数据集)和语言任务(IOI、IOI-Hard、Greater-Than)。结果表明,认证电路在准确率上最高提升56%,组件数量最多减少80%,并在基线退化的场景中仍保持可靠。该工作将电路发现置于形式化基础之上,生成可证明稳定且更贴近目标概念的机制解释。代码:https://github.com/AlaaAnani/certified-circuits。
原文摘要 · Abstract (English)
Understanding how neural networks arrive at their predictions is essential for debugging, auditing, and deployment. Mechanistic interpretability pursues this goal by identifying circuits--minimal subnetworks responsible for specific behaviors. However, existing circuit discovery methods are brittle: circuits depend strongly on the chosen concept dataset and often fail to transfer out-of-distribution, raising doubts whether they capture the concept or merely dataset-specific artifacts. We introduce Certified Circuits, which provide provable stability guarantees for circuit discovery. Our framework wraps any black-box discovery algorithm with randomized data subsampling to certify that inclusion decisions over circuit components--neurons or edges of the model graph, depending on the base algorithm--are invariant to bounded edit-distance perturbations of the concept dataset. Unstable components are abstained from, yielding circuits that are more compact and more accurate. We validate across three architectures (ResNet, ViT, GPT-2) on vision (ImageNet and four OOD datasets) and language (IOI, IOI-Hard, Greater-Than) tasks. Certified circuits achieve up to 56% higher accuracy and up to 80% fewer components, and remain reliable where baselines degrade. Certified Circuits puts circuit discovery on formal ground by producing mechanistic explanations that are provably stable and better aligned with the target concept. Code: https://github.com/AlaaAnani/certified-circuits.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。