用信息瓶颈方法端到端发现更准确的模型电路
IBCircuit: Towards Holistic Circuit Discovery with Information Bottleneck
- 基于信息瓶颈原理，无需人为设计干扰激活
- 在IOI和大于比较任务中发现更小更准的电路组件
- 适合想理解大模型内部工作机制的研究者
电路发现近年成为解释语言模型非平凡行为的潜在方向，旨在找出模型中负责特定任务的计算子图（即电路）。然而，现有研究大多忽视了电路的整体性，且需为不同任务设计特定的破坏性激活，过程不准确且效率低。本文提出一种基于信息瓶颈原理的端到端方法IBCircuit，实现整体性电路发现。该方法是一个优化框架，可适用于任意给定任务，无需繁琐的激活破坏设计。在间接宾语识别（IOI）和大于比较任务中，IBCircuit相比近期相关工作发现了更忠实、更精简的关键节点与边组件。
原文摘要 · Abstract (English)
Circuit discovery has recently attracted attention as a potential research direction to explain the non-trivial behaviors of language models. It aims to find the computational subgraphs, also known as circuits, within the model that are responsible for solving specific tasks. However, most existing studies overlook the holistic nature of these circuits and require designing specific corrupted activations for different tasks, which is inaccurate and inefficient. In this work, we propose an end-to-end approach based on the principle of Information Bottleneck, called IBCircuit, to identify informative circuits holistically. IBCircuit is an optimization framework for holistic circuit discovery and can be applied to any given task without tediously corrupted activation design. In both the Indirect Object Identification (IOI) and Greater-Than tasks, IBCircuit identifies more faithful and minimal circuits in terms of critical node components and edge components compared to recent related work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。