揭示大模型回答具体问题时的内部信息路径,让解释更精准可信。
Query Circuits: Explaining How Language Models Answer User Prompts
- 直接追踪输入到输出的信息流,找到特定问答的内部电路。
- 仅用1.3%的连接就能恢复MMLU任务60%的性能表现。
- 适合想理解模型决策过程的研究者和开发者。
解释语言模型为何对特定输入产生特定输出,需要局部、基于输入层面的解释。现有方法只能发现全局能力电路(如间接宾语识别),无法说明模型为何以特定方式回应具体查询。本文提出查询电路,直接追踪模型内部将特定输入映射到输出的信息流。不同于依赖代理的方法(如稀疏自编码器),查询电路在模型内部识别,解释更准确且计算更高效。为提升实用性,本文提出归一化偏差忠实度(NDF)作为评估指标,广泛适用于各类电路发现;并设计采样方法,高效找出稀疏但忠实反映模型行为的电路。在多个基准测试(IOI、算术、MMLU、ARC)中,发现模型内存在极稀疏的查询电路,可恢复大部分单个查询的表现。例如,仅覆盖1.3%模型连接的电路即可恢复约60%的MMLU答题性能。总体而言,查询电路为实现可信赖、可扩展的个体输入解释迈出了关键一步。
原文摘要 · Abstract (English)
Explaining why a language model produces a particular output requires local, input-level explanations. Existing methods uncover global capability circuits (e.g., indirect object identification), but not why the model answers a specific input query in a particular way. We introduce query circuits, which directly trace the information flow inside a model that maps a specific input to the output. Unlike surrogate-based approaches (e.g., sparse autoencoders), query circuits are identified within the model itself, resulting in more faithful and computationally accessible explanations. To make query circuits practical, we address two challenges. First, we introduce Normalized Deviation Faithfulness (NDF), a robust metric to evaluate how well a discovered circuit recovers the model's decision for a specific input, and is broadly applicable to circuit discovery beyond our setting. Second, we develop sampling-based methods to efficiently identify circuits that are sparse yet faithfully describe the model's behavior. Across benchmarks (IOI, arithmetic, MMLU, and ARC), we find that there exist extremely sparse query circuits within the model that can recover much of its performance on single queries. For example, a circuit covering only 1.3% of model connections can recover about 60% of performance on an MMLU questions. Overall, query circuits provide a step towards faithful, scalable explanations of how language models process individual inputs. The project page is at https://tony10101105.github.io/query-circuit/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。