arXiv:2605.22462cs.CLcs.AI2026-05

提出五阶段方法,揭示Transformer模型中特征的因果作用

From Correlation to Cause: A Five-Stage Methodology for Feature Analysis in Transformer Language Models

论文配图:From Correlation to Cause: A Five-Stage Methodology for Feature Analysis in Transformer Language Models
图 1 · 摘自论文原文
  • 分五步分析特征:探测、提取、验证、测试、部署集成
  • 发现15个关键特征仅部分因果,98%任务准确率不受影响
  • 方法可定位有效监控策略,成本降低99.1%适合安全应用

我们提出一种针对Transformer语言模型的五阶段因果特征分析方法(探针设计、特征提取、因果验证、鲁棒性测试与部署集成),并在GPT-2 small上完成端到端演示,用于间接宾语识别(IOI)任务。激活补丁法恢复了标准的IOI电路(第9层第9头单独即可恢复+1.02准确率)。稀疏自编码器恢复出每名实体选择性特征,效应大小为30至50个激活单元。因果验证表明这些特征具有特定性但仅部分因果:剔除其中15个,模型在98%提示下仍保持准确。两项受NLA启发的评估强化此结论:这15个特征仅解释31%激活方差(对比SAE的99.7%),选择性比率与因果强度负相关(r = -0.56)。在三种分布偏移下的鲁棒性测试显示,电路可稳定迁移,但特征剔除效果显著下降,暴露检测鲁棒性与因果鲁棒性之间的差距。基于成本的部署评估(假设误报损失$50/次,误检成本$0.42/次,2%错误率)发现最优监控配置每千次查询成本为$8.96,相比基准$1000降低99.1%。最优组合策略随成本比和基线率变化。各阶段协同带来单一阶段无法实现的洞见。

原文摘要 · Abstract (English)

We propose a five-stage methodology for causal feature analysis in transformer language models (probe design, feature extraction, causal validation, robustness testing, and deployment integration) and demonstrate it end-to-end on GPT-2 small performing the Indirect Object Identification (IOI) task. Activation patching recovers the canonical IOI circuit (layer-9 head 9 alone gives recovery +1.02). A sparse autoencoder recovers per-name selective features with effect sizes of 30 to 50 activation units. Causal validation finds these features specifically but only partially causal: ablating fifteen of them leaves the model accurate on 98% of prompts. Two NLA-inspired evaluations strengthen this picture: the fifteen selective features explain only 31% of activation variance versus the SAE's 99.7%, and selectivity ratio anticorrelates with causal force (r = -0.56). Robustness testing under three distribution shifts finds that the circuit transfers cleanly but feature ablation effects degrade substantially, exposing a gap between detection robustness and causal robustness. A cost-based deployment evaluation (assumed $50/FN, $0.42/FP, 2% error rate) finds an optimal monitor configuration yielding $8.96 per 1000 queries against a $1000 baseline, a 99.1% saving. Optimal composition strategy varies with cost ratio and base rate. The conjunction of stages produces findings no single stage would.

因果分析Transformer特征提取模型监控

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。