分析GPT-2在间接宾语识别任务中失败时的稀疏特征激活，发现特定神经元与失败高度相关。
Reading Task Failure Off the Activations: A Sparse-Feature Audit of GPT-2 Small on Indirect Object Identification
- 用稀疏自编码器分析模型激活，定位在失败案例中异常活跃的特征。
- 特征17491在'钥匙'提示下激活最强，此时模型失败率达93.3%（其他对象仅7.5%）。
- 方法可复现且无需高端设备，适合关注模型内部机制的研究者。
我们对GPT-2 small在间接宾语识别（IOI）任务中的失败与成功样本进行了一次可复现的小规模审计，分析其第8层残差流中稀疏自编码器（SAE）特征的激活差异。在300个提示下，模型准确率为79.7%；从Bloom（2024）发布的24,576个特征中，146个通过了霍尔姆校正显著性检验，105个达到大效应量（|Cohen's d| > 0.8）。最强失败相关特征为特征17491（d=+2.93），Neuronpedia标签为'cryptographic keys'，仅在提示中的转移宾语为'钥匙'时显著激活，此时模型失败率达93.3%，其余七类对象仅7.5%（费希尔精确检验p = 8.79 × 10^-33）。对该特征进行了三项验证：(i) 因果消融（置零该特征）未恢复准确率（6.7% → 4.4%），表明其非充分原因；(ii) 原始768维残差流的逻辑回归已达ROC AUC 0.929，与前100个SAE特征相当，说明SAE仅提升可解释性；(iii) 五次随机种子实验中，'钥匙'子集失败率稳定在75.0%–93.3%，但该特征仅为一次运行中的最强相关项。因此，方法论贡献在于轻量、模型无关的审计流水线，而非单个特征。代码、数据集、激活矩阵、消融脚本及图表均已公开，全流程可在笔记本（Apple M3 Max，无独立显卡）上运行。
原文摘要 · Abstract (English)
We report a small, reproducible audit of which sparse-autoencoder (SAE) features of GPT-2 small fire differently on failed versus successful trials of the Indirect Object Identification (IOI) task. On 300 prompts, GPT-2 small reaches 79.7% accuracy; 146 of the 24,576 features in the layer-8 residual-stream SAE release of Bloom (2024) clear a Holm-corrected significance threshold and 105 reach a large effect size (|Cohen's d| > 0.8). The strongest single correlate of failure -- feature 17,491, d=+2.93, Neuronpedia label 'cryptographic keys' -- is essentially silent except when the prompt's transferred object is 'the keys,' on which GPT-2 small fails 93.3% of the time vs. 7.5% on the other seven objects (Fisher exact p = 8.79 x 10^-33). We put this correlate through three controls that a mechanistic claim should pass. (i) A causal ablation: zeroing feature 17,491 in the residual stream across all token positions of the 45 keys prompts does not restore accuracy (6.7% -> 4.4%); the feature is a correlate, not a sufficient cause at this layer. (ii) A representation baseline: a logistic regression on the raw 768-dimensional residual stream reaches 5-fold ROC AUC = 0.929, matching the top-100 SAE features (0.927); the SAE basis adds interpretability, not predictive power. (iii) A seed-robustness check: across five random seeds the keys-subset failure rate stays in 75.0--93.3% (the behavioural effect is real), but feature 17,491 is the top-|d| feature in only 1 of 5 runs. The methodological contribution is therefore the audit pipeline (cheap, model-agnostic, surfaces named correlates) rather than any single feature found through it. We release the code, the 300-prompt corpus, the 300x24,576 activation matrix, the ablation and baseline scripts, and the figures. The full pipeline runs on a laptop (Apple M3 Max, no discrete GPU).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。