追踪大模型训练中组件演化的机制，揭示知识如何逐步形成。
Time Course MechInterp: Analyzing the Evolution of Components and Knowledge in Large Language Models
- 按角色分类注意力头和前馈网络，追踪其在预训练中的变化
- 初期依赖通用组件，后期逐渐专业化，部分组件被重新利用
- 注意力头更易更换，位置关系比名称关系更早达到高准确率
理解大语言模型（LLMs）如何获取和存储事实知识，对提升其可解释性和可靠性至关重要。本文通过追踪OLMo-7B模型在预训练过程中注意力头和前馈网络（FFNs）的角色演变，分析其知识表征的演化过程。我们将这些组件分为四类：通用、实体、关系-答案特定和事实-答案特定，并考察其稳定性与转换情况。结果显示，模型初期依赖广泛通用的组件，随着训练推进逐渐专业化；一旦模型能稳定预测答案，部分组件会被重用，表明存在自适应学习过程。值得注意的是，注意力头的替换率最高，而前馈网络在整个训练中保持更高稳定性。探针实验还显示，基于位置的关系比基于名称的关系更早收敛至高准确率，揭示了任务复杂度对知识获取动态的影响。这些发现为大模型知识形成的机制提供了深入洞见。
原文摘要 · Abstract (English)
Understanding how large language models (LLMs) acquire and store factual knowledge is crucial for enhancing their interpretability and reliability. In this work, we analyze the evolution of factual knowledge representation in the OLMo-7B model by tracking the roles of its attention heads and feed forward networks (FFNs) over the course of pre-training. We classify these components into four roles: general, entity, relation-answer, and fact-answer specific, and examine their stability and transitions. Our results show that LLMs initially depend on broad, general-purpose components, which later specialize as training progresses. Once the model reliably predicts answers, some components are repurposed, suggesting an adaptive learning process. Notably, attention heads display the highest turnover. We also present evidence that FFNs remain more stable throughout training. Furthermore, our probing experiments reveal that location-based relations converge to high accuracy earlier in training than name-based relations, highlighting how task complexity shapes acquisition dynamics. These insights offer a mechanistic view of knowledge formation in LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。