用机制可解释性方法挖掘图神经网络的算法计算电路。
MINAR: Mechanistic Interpretability for Neural Algorithmic Reasoning
- 将归因打补丁法移植到GNN，实现神经算法推理的电路发现。
- 在两个案例中成功恢复出与经典算法对齐的神经电路。
- 揭示多任务训练下电路复用规律，适合研究模型内部机制者阅读。
神经算法推理（NAR）研究图神经网络（GNNs）模拟贝尔曼-福特等经典算法的能力，即算法对齐现象。与此同时，大语言模型（LLMs）的发展推动了机制可解释性研究，旨在识别执行特定计算的细粒度模型组件（如电路）。本文提出神经算法推理的机制可解释性（MINAR），一个高效的电路发现工具箱，将机制可解释性中的归因打补丁方法适配至GNN场景。通过两个案例研究，我们证明MINAR能够从训练于算法任务的GNN中恢复出忠实的神经元级电路。研究揭示了训练过程中电路形成与剪枝的机制，并为并行训练多个任务时，相关任务间电路组件的复用提供了新见解。代码已开源：https://github.com/pnnl/MINAR。
原文摘要 · Abstract (English)
The recent field of neural algorithmic reasoning (NAR) studies the ability of graph neural networks (GNNs) to emulate classical algorithms like Bellman-Ford, a phenomenon known as algorithmic alignment. At the same time, recent advances in large language models (LLMs) have spawned the study of mechanistic interpretability, which aims to identify granular model components like circuits that perform specific computations. In this work, we introduce Mechanistic Interpretability for Neural Algorithmic Reasoning (MINAR), an efficient circuit discovery toolbox that adapts attribution patching methods from mechanistic interpretability to the GNN setting. We show through two case studies that MINAR recovers faithful neuron-level circuits from GNNs trained on algorithmic tasks. Our study sheds new light on the process of circuit formation and pruning during training, as well as giving new insight into how GNNs trained to perform multiple tasks in parallel reuse circuit components for related tasks. Our code is available at https://github.com/pnnl/MINAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。