系统分析大模型训练各阶段动态，揭示增益与瓶颈。
EvoLM: In Search of Lost Language Model Training Dynamics
- 构建多阶段训练框架，统一评估预训练到强化学习全过程
- 100多个1B/4B模型实验显示过度预训练收益递减
- 适合关注模型训练优化与可复现研究的开发者
现代语言模型训练已分化为多个阶段，使下游开发者难以评估各阶段设计选择的影响。我们提出EvoLM，一套支持系统性、透明化分析语言模型从预训练、持续预训练、监督微调到强化学习各阶段训练动态的模型体系。我们从头训练了超过100个参数量为1B和4B的语言模型，并评估其上游（语言建模）与下游（问题求解）能力，涵盖域内与域外泛化表现。关键发现包括：过度预训练和后训练存在边际收益递减；在领域特定的持续预训练中需有效缓解遗忘；持续预训练在连接预训练与后训练阶段中起关键作用；配置监督微调与强化学习时存在多种复杂权衡。为促进开放研究与可复现性，我们发布所有预训练及后训练模型、各阶段训练数据集，以及完整的训练与评估流水线。
原文摘要 · Abstract (English)
Modern language model (LM) training has been divided into multiple stages, making it difficult for downstream developers to evaluate the impact of design choices made at each stage. We present EvoLM, a model suite that enables systematic and transparent analysis of LMs' training dynamics across pre-training, continued pre-training, supervised fine-tuning, and reinforcement learning. We train over 100 LMs with 1B and 4B parameters from scratch, and evaluate both upstream (language modeling) and downstream (problem-solving) capabilities, including considerations of both in-domain and out-of-domain generalization. Key insights highlight the diminishing returns from excessive pre-training and post-training, the importance and practices of mitigating forgetting during domain-specific continued pre-training, the crucial role of continued pre-training in bridging pre-training and post-training phases, and various intricate trade-offs when configuring supervised fine-tuning and reinforcement learning. To facilitate open research and reproducibility, we release all pre-trained and post-trained models, training datasets for all stages, and our entire training and evaluation pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。