发现GPT-2小模型末层存在可读的异常处理机制。
Darkness Visible: Reading the Exception Handler of a Language Model
- 末层MLP由27个命名神经元构成三层异常处理结构。
- 5个共识神经元监控语言维度，4/7至5/7间干预由益转害。
- 知识神经元实为路由工具，非事实存储，适配语言建模研究者。
GPT-2 Small的末层MLP呈现出完全可读的路由程序——27个命名神经元组成三层异常处理结构，而其所路由的知识仍缠绕于约3040个残差神经元中。我们以数值精度将全部3072个神经元分解为：5个融合核心神经元用于重置词汇至功能词，10个区分器抑制错误候选，5个专家神经元检测结构边界，以及7个共识神经元分别监控不同语言维度。共识-异常交叉点（即MLP干预由有益转为有害）在统计上极为清晰（置信区间95%排除零值；交叉发生在4/7与5/7之间）。三项实验表明，该模型第11层的‘知识神经元’（Dai et al., 2022）实为路由基础设施，而非事实存储：其通过放大或抑制注意力产生的残差流信号实现功能，且强度随上下文约束增强。花园路径实验揭示反向花园路径效应——GPT-2立即使用动词子分类，表明异常处理机制作用于词元级可预测性，而非句法结构。此架构仅在终端层显现；在更深模型中，我们预测等效结构出现在末层而非第11层。代码与数据：https://github.com/pbalogh/transparent-gpt2
原文摘要 · Abstract (English)
The final MLP of GPT-2 Small exhibits a fully legible routing program -- 27 named neurons organized into a three-tier exception handler -- while the knowledge it routes remains entangled across ~3,040 residual neurons. We decompose all 3,072 neurons (to numerical precision) into: 5 fused Core neurons that reset vocabulary toward function words, 10 Differentiators that suppress wrong candidates, 5 Specialists that detect structural boundaries, and 7 Consensus neurons that each monitor a distinct linguistic dimension. The consensus-exception crossover -- where MLP intervention shifts from helpful to harmful -- is statistically sharp (bootstrap 95% CIs exclude zero at all consensus levels; crossover between 4/7 and 5/7). Three experiments show that "knowledge neurons" (Dai et al., 2022), at L11 of this model, function as routing infrastructure rather than fact storage: the MLP amplifies or suppresses signals already present in the residual stream from attention, scaling with contextual constraint. A garden-path experiment reveals a reversed garden-path effect -- GPT-2 uses verb subcategorization immediately, consistent with the exception handler operating at token-level predictability rather than syntactic structure. This architecture crystallizes only at the terminal layer -- in deeper models, we predict equivalent structure at the final layer, not at layer 11. Code and data: https://github.com/pbalogh/transparent-gpt2
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。