揭示GPT模型总结任务中的核心工作机制
Mechanistic Interpretability of GPT-like Models on Summarization Tasks
- 通过对比预训练与微调模型，定位总结任务的关键神经通路
- 中层（2、3、5层）62%注意力头熵降低，体现信息聚焦变化
- 针对性微调关键通路可提升性能，且训练更高效
机制可解释性研究旨在揭示大语言模型的内部运作机制，但现有工作多集中于分类或生成任务，缺乏对总结任务的关注。本文提出一种分析GPT类模型适应总结任务的可解释性框架。通过对比预训练与微调模型，量化注意力模式与内部激活的变化。识别出经历显著转变的特定层和注意力头，定位到模型架构中的“总结电路”。研究发现，中层（尤其是第2、3、5层）变化最显著，62%的注意力头熵降低，表明信息选择趋向聚焦。我们证明，针对这些识别出的电路进行定向LoRA微调，在更少训练轮次下即可实现优于标准LoRA微调的性能。该工作弥合了黑箱评估与机制理解之间的差距，揭示了神经网络在总结过程中进行信息选择与压缩的内在机制。
原文摘要 · Abstract (English)
Mechanistic interpretability research seeks to reveal the inner workings of large language models, yet most work focuses on classification or generative tasks rather than summarization. This paper presents an interpretability framework for analyzing how GPT-like models adapt to summarization tasks. We conduct differential analysis between pre-trained and fine-tuned models, quantifying changes in attention patterns and internal activations. By identifying specific layers and attention heads that undergo significant transformation, we locate the "summarization circuit" within the model architecture. Our findings reveal that middle layers (particularly 2, 3, and 5) exhibit the most dramatic changes, with 62% of attention heads showing decreased entropy, indicating a shift toward focused information selection. We demonstrate that targeted LoRA adaptation of these identified circuits achieves significant performance improvement over standard LoRA fine-tuning while requiring fewer training epochs. This work bridges the gap between black-box evaluation and mechanistic understanding, providing insights into how neural networks perform information selection and compression during summarization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。