CPT中损失下降不等于知识真正掌握，学习过程极不稳定。
What Does Loss Optimization Actually Teach, If Anything? Knowledge Dynamics in Continual Pre-training of LLMs
- 在持续预训练中插入诊断探针，实时监测知识获取与模型能力变化。
- 损失持续下降，但事实知识学习波动大，早期就出现泛化能力退化。
- 知识路径快速重配置，解释了为何知识窗口窄且易遗忘，适合研究者参考。
持续预训练（CPT）广泛用于大型语言模型的事实知识更新。当前做法将损失视为知识学习的代理指标，却缺乏对训练过程中知识动态变化的理解。本文将CPT视为知识学习过程而非单纯优化问题，构建了分布匹配的事实文档基准，并在训练循环中嵌入诊断探针，实现对知识获取动态及域外（OOD）能力（如数学）变化的逐轮测量。分析显示，尽管损失单调下降，但事实知识学习呈现不稳定性与非单调性；已学知识极少巩固，学习高度依赖先前暴露，且域外性能从早期即开始下降。电路分析揭示知识通路在各轮次间快速重构，解释了狭窄的知识获取窗口与系统性遗忘现象。结果表明，损失优化与学习进展在CPT中严重错位，提示应基于任务级学习动态评估停止时机。
原文摘要 · Abstract (English)
Continual Pre-Training (CPT) is widely used for acquiring and updating factual knowledge in LLMs. This practice treats loss as a proxy for knowledge learning, while offering no grounding into how it changes during training. We study CPT as a knowledge learning process rather than a solely optimization problem. We construct a controlled, distribution-matched benchmark of factual documents and interleave diagnostic probes directly into the CPT loop, enabling epoch-level measurement of knowledge acquisition dynamics and changes in Out-Of-Domain (OOD) general skills (e.g., math). We further analyze how CPT reshapes knowledge circuits during training. Across three instruction-tuned LLMs and multiple CPT strategies, optimization and learning systematically diverge as loss decreases monotonically while factual learning is unstable and non-monotonic. Acquired facts are rarely consolidated, learning is strongly conditioned on prior exposure, and OOD performance degrades from early epochs. Circuit analysis reveals rapid reconfiguration of knowledge pathways across epochs, providing an explanation for narrow acquisition windows and systematic forgetting. These results show that loss optimization is misaligned with learning progress in CPT and motivate evaluation of stopping criteria based on task-level learning dynamics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。