实验证明GPT-2的情感分析靠早期词义、晚期统一整合，非中间分层处理。
Mechanistic Interpretability of GPT-2: Lexical and Contextual Layers in Sentiment Analysis
- 通过激活替换法逐层测试，发现前3层负责词级情感识别。
- 中层无专门整合功能，否定、反语等复杂现象在后4层统一处理。
- 揭示大模型情感计算机制，适合研究模型可解释性的人参考。
我们对GPT-2进行了机制可解释性研究，通过系统性地在12个层级进行激活替换，因果检验情感信息在各层的处理方式。假设存在两阶段结构：早期词汇检测与中期上下文整合。实验确认前3层（0-3）作为词汇级情感探测器，编码稳定且位置敏感的情感信号，基本不受上下文影响。然而，三种中期整合假说——中层集中、现象特异性、分布式处理均被证伪。相反，否定、讽刺、领域转移等上下文现象主要在后4层（8-11）通过统一、非模块化机制完成整合。这一因果证据表明，GPT-2的情感计算并不遵循预想的层级结构，凸显了对大语言模型上下文整合机制进行进一步实证研究的必要性。
原文摘要 · Abstract (English)
We present a mechanistic interpretability study of GPT-2 that causally examines how sentiment information is processed across its transformer layers. Using systematic activation patching across all 12 layers, we test the hypothesized two-stage sentiment architecture comprising early lexical detection and mid-layer contextual integration. Our experiments confirm that early layers (0-3) act as lexical sentiment detectors, encoding stable, position specific polarity signals that are largely independent of context. However, all three contextual integration hypotheses: Middle Layer Concentration, Phenomenon Specificity, and Distributed Processing are falsified. Instead of mid-layer specialization, we find that contextual phenomena such as negation, sarcasm, domain shifts etc. are integrated primarily in late layers (8-11) through a unified, non-modular mechanism. These experimental findings provide causal evidence that GPT-2's sentiment computation differs from the predicted hierarchical pattern, highlighting the need for further empirical characterization of contextual integration in large language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。