arXiv:2607.00089cs.LG2026-07

提出可查询的结构化表示协议,让神经网络组件分析结果更易复用。

Representation as a Bottleneck for Mechanistic Interpretability: The Manifestation Unit Protocol

论文配图:Representation as a Bottleneck for Mechanistic Interpretability: The Manifestation Unit Protocol
图 1 · 摘自论文原文
  • 设计结构化数据协议(E,S,R,D,G,T),自动组织组件统计信息
  • 在生成、判别与语言模型中均实现高效检索,优于无结构基线
  • 支持下游审计与干预,适用于需要可解释性的模型开发者

机制可解释性研究积累了大量关于神经网络组件编码与交互的细粒度分析,但这些成果难以复用:选择性表格、电路图和特征列表仍被锁在各研究笔记中,无法组合、不支持自然语言查询,也无法直接用于下游审计或干预。本文将组件分析与下游应用之间的表示层视为瓶颈,独立评估并引入「显现单元协议」(Manifestation Units),一种包含五元组(E, S, R, D, G)并扩展注意力头原语(T)的类型化元组协议,将组件统计信息结构化存储,支持混合检索。该协议在生成视觉(beta-VAE)、判别视觉(CNN)和语言(GPT-2)模型中实例化,结果显示:类型化结构显著优于无结构基线;在预算匹配条件下,检索到的CNN滤波器满足因果充分性与必要性标准。协议无需修改即可吸收注意力头原语,在预算匹配控制下能准确恢复已知的IOI电路成员,并揭示一个不可约的双字段核心(S+R),其余字段要么冗余,要么产生干扰。本工作定位为机制可解释性的架构基础设施,而非前沿验证。

原文摘要 · Abstract (English)

Mechanistic interpretability has produced a rich inventory of component-level analyses that characterise what neural-network components encode and how they interact. Their outputs, however, are not easily reusable: selectivity tables, circuit diagrams, and feature lists remain locked in per-study notebooks - non-composable, not queryable in natural language, and not directly actionable for downstream audit or intervention. We study the representation layer that sits between these analyses and downstream use as a bottleneck that can be evaluated independently, and introduce Manifestation Units, a typed tuple protocol (E, S, R, D, G) extended with attention-head primitives (T) for transformer architectures, organising per-component statistics into structured fields populated automatically and queried through hybrid retrieval. Instantiated across generative vision (beta-VAE), discriminative vision (CNN), and language (GPT-2), the protocol supports two findings: typed structure substantially outperforms unstructured baselines on retrieval, and CNN filters retrieved by the schema satisfy causal sufficiency and necessity criteria under matched-budget controls. The schema absorbs attention-head primitives without modification, set-recovers known IOI circuit members under retrieval-budget-matched controls, and reveals an irreducible two-field core (S+R) with remaining fields either redundant or actively interfering. We present this as schema infrastructure for mechanistic interpretability rather than frontier-scale validation.

可解释性结构化数据神经网络协议设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。