让数据自己揭示语言模型的内在计算机制,而非依赖人工定义的任务边界。
Data-driven Circuit Discovery for Interpretability of Language Models

- 基于数据聚类发现多组相似处理模式,为每组生成独立电路
- 同一数据集内可发现多个高保真度电路,优于传统单一电路
- 适合研究模型内部机制、解释性分析的研究者
电路发现旨在通过定位和解释一个负责语言模型行为的计算子图(即电路),来阐明模型如何完成特定任务。现有方法为假设驱动:先以数据集非正式定义任务,再在该数据集上运行算法得到单一电路。这隐含两个强假设:模型用单一电路实现任务,且数据集能充分代表人类理解的任务。我们系统测试了四个已研究任务,发现即使数据集语义不变,微小变化也会导致电路边重叠率低、跨数据集保真度差。更显著的是,在混合了两个不同任务的数据集上,尽管分别发现的电路跨任务保真度接近零,现有方法仍返回一个对两个任务均高保真的单一电路。表明当前方法发现的是数据特异性电路,而非通用任务电路。为此,我们提出数据驱动电路发现(DCD)框架,放弃上述两个假设:不返回单一电路,而是先按模型处理相似性对数据聚类,再为每组发现独立电路。这使不同机制得以分离呈现,每条电路仅解释其对应组,而非整个任务。实验显示,DCD在每个数据集中发现多个电路,每条均比传统方法发现的单个电路对所属组更具保真度。总体而言,DCD让数据揭示模型内部的机制结构,而非依赖可能与模型计算组织不符的人工任务边界。
原文摘要 · Abstract (English)
Circuit discovery aims to explain how language models (LMs) implement a specific task by localizing and interpreting a circuit, a computational subgraph responsible for the LM's behavior. Existing circuit discovery methods are hypothesis-driven; they first informally define a task with a dataset, and then apply a circuit discovery algorithm over that dataset to obtain a single circuit. This imposes two strong assumptions: that the LM implements the task with a single circuit, and that the dataset adequately represents the task as humans understand it. We systematically test these assumptions across four previously studied tasks and find that even minor dataset variations that preserve task semantics can produce circuits with low edge overlap and cross-dataset faithfulness. More strikingly, when applied to a mixed dataset with two distinct tasks whose separately discovered circuits have near-zero cross-faithfulness, existing methods still return a single circuit with high faithfulness across both tasks. This indicates that current methods discover dataset-specific circuits, rather than general task circuits. We propose Data-driven Circuit Discovery (DCD), a new discovery framework that drops both assumptions: instead of returning a single circuit for a dataset, DCD first clusters examples in the dataset by how similarly the model processes them and discovers a separate circuit for each group. This allows distinct mechanisms to appear separately rather than merged into a single circuit; each circuit explains its group, not the full task. Experiments show that DCD discovers multiple circuits per dataset, each more faithful to its group than a single circuit discovered by existing methods. Broadly, DCD lets the data reveal mechanistic structure within LMs, rather than relying on human-defined task boundaries that may not align with how models organize their computation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。