arXiv:2603.12423cs.CL2026-03中稿 · the 2026 IEEE 16th…

分析GPT-2如何处理否定,发现关键注意力头集中在中层。

Interpreting Negation in GPT-2: Layer- and Head-Level Causal Analysis

  • 通过激活修补和注意力头消融,追踪否定信号在模型中的传播路径。
  • 仅4到6层的少数注意力头决定否定敏感度,其缺失使模型误判率上升。
  • 结果在多种否定形式和外部数据集上一致,适合研究模型可解释性的人参考。

否定仍是现代语言模型的持续挑战,常导致语义反转或事实错误。本文对GPT-2 Small内部如何处理此类语言转换进行了因果分析,考察其在层与注意力头层面的隐藏表示。基于自建的1.2万对匹配的肯定句与否定句数据集(覆盖多种语言模板与否定形式),定义了衡量模型区分能力的否定效应得分(NES)。通过两种干预手段:激活修补(将肯定句内部激活插入否定句)与注意力头消融(临时禁用特定头),揭示了否定信号在模型中的演化过程。结果表明,该能力高度集中于第4至第6层的少数中层注意力头;消融这些头会显著降低模型否定敏感度(域内测试中NES升高),而恢复肯定激活则进一步提升NES,证明这些头携带肯定信号而非恢复基线行为。在外部数据集xNot360上,消融轻微降低NES,且恢复性能超过基线,说明该因果模式在不同否定形式中保持一致,尽管幅度较小。

原文摘要 · Abstract (English)

Negation remains a persistent challenge for modern language models, often causing reversed meanings or factual errors. In this work, we conduct a causal analysis of how GPT-2 Small internally processes such linguistic transformations. We examine its hidden representations at both the layer and head level. Our analysis is based on a self-curated 12,000-pair dataset of matched affirmative and negated sentences, covering multiple linguistic templates and forms of negation. To quantify this behavior, we define a metric, the Negation Effect Score (NES), which measures the model's sensitivity in distinguishing between affirmative statements and their negations. We carried out two key interventions to probe causal structure. In activation patching, internal activations from affirmative sentences were inserted into their negated counterparts to see how meaning shifted. In ablation, specific attention heads were temporarily disabled to observe how logical polarity changed. Together, these steps revealed how negation signals move and evolve through GPT-2's layers. Our findings indicate that this capability is not widespread; instead, it is highly concentrated within a limited number of mid-layer attention heads, primarily within layers 4 to 6. Ablating these specific components directly disrupts the model's negation sensitivity: on our in-domain, ablation increased NES (indicating weaker negation sensitivity), and re-introducing cached affirmative activations (rescue) increased NES further, confirming that these heads carry affirmative signal rather than restoring baseline behavior. On xNot360, ablation slightly decreased NES and rescue restored performance above baseline. This pattern demonstrates that these causal patterns are consistent across various negation forms and remain detectable on the external xNot360 benchmark, though with smaller magnitude.

模型可解释性否定理解注意力机制因果分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。