一个探针无法确定Mamba-2中真正负责计算的神经元,结果不唯一且机制复杂。
A Circuit, Not The Circuit: Non-Unique Causal Localisation of the Mamba-2 State Sink
- 用探针定位关键神经元时,发现多组不同头共享相似激活但功能不同
- 同一头部在输入与输出端干预,损失变化差异巨大(0.16 vs 1.5-7.3 nats)
- 虽无法锁定确切电路,但这些头对行为仍至关重要,影响任务成功率
机械可解释性通常通过探针识别最高激活单元并将其视为执行计算的电路。我们在Mamba模型的state sink上测试这一方法:该组件是Transformer注意力机制中类似‘焦点’的选通结构,其Delta-gate在边界标记(如BOS、换行符)处显著激活。在Mamba-1的通道粒度下,探针识别出的单元具有因果效应;但在Mamba-2的头粒度下,标签与定位之间的关联以三种方式断裂:因果集合不唯一——一组几乎无重叠的顶级激活头,能匹配或超过原有探针的消融效果;表征不反映功能——约四分之一至三分之一的头对边界符号反应相似,却无法复制专业头的因果特征;干预表面不同导致效果分离——扰动输入依赖的Delta仅使损失变化≤0.16 nats,而抑制相同头的输出则使损失变化达1.5–7.3 nats。尽管如此,这些头在行为上仍重要,在两种旗舰检查点上,将上下文长度设为1024时,扰动后针取任务成功率从30/30降至0/30。因此,在选择性状态空间模型中,单一探针只能识别一个有效头集,而非唯一电路。
原文摘要 · Abstract (English)
Mechanistic interpretability routinely reads a probe and labels its top-activating units as the circuit executing the computation. We test the move in Mamba, on the state sink: the selective state-space analogue of the Transformer attention sink, where the Delta-gate fires disproportionately on boundary tokens such as BOS and newline. At Mamba-1 channel granularity the probe's units carry the causal effect. At Mamba-2 head granularity the label-to-locus link breaks in three ways. The causal set is not unique: a near-disjoint set of top-activation heads, sharing almost none of the specialists, matches or exceeds their ablation effect. Representation does not track function: dual heads, a quarter to over a third of all heads, respond alike to BOS and newline yet do not reproduce the specialists' causal profile. And the effect dissociates by intervention surface: perturbing the input-dependent Delta the probe reads moves the loss by at most 0.16 nats, while suppressing the same heads' output moves it by 1.5 to 7.3 nats. The heads still matter behaviourally, dropping needle-retrieval success on a RULER-style probe from 30/30 to 0/30 at 1024 context on both flagship checkpoints. A representational signature in a selective state-space model thus marks a behaviourally important head set without pinning down the circuit: a single probe recovers a circuit, not the circuit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。