预训练越久，越难微调，反而影响最终效果。
Overtrained Language Models Are Harder to Fine-Tune
- 发现过度预训练使模型参数对微调更敏感，导致性能下降。
- 3T token预训练的OLMo-1B在多个基准上比2.3T版本差超2%。
- 适用于关注模型可微调性与预训练策略的科研人员。
大语言模型通常在不断增长的语料上进行预训练，普遍认为更好的预训练表现能带来更优的下游性能。本文挑战这一假设，揭示延长预训练会使模型更难微调，导致最终性能下降，称此现象为‘灾难性过训练’。例如，基于3万亿标记符预训练的指令微调版OLMo-1B模型，在多个标准大模型评测基准上表现比2.3万亿标记符版本差超过2%。通过受控实验与理论分析，我们发现灾难性过训练源于预训练参数对修改（包括微调）的系统性敏感度上升。研究呼吁重新审视预训练设计，需兼顾模型的下游适应能力。
原文摘要 · Abstract (English)
Large language models are pre-trained on ever-growing token budgets under the assumption that better pre-training performance translates to improved downstream models. In this work, we challenge this assumption and show that extended pre-training can make models harder to fine-tune, leading to degraded final performance. We term this phenomenon catastrophic overtraining. For example, the instruction-tuned OLMo-1B model pre-trained on 3T tokens leads to over 2% worse performance on multiple standard LLM benchmarks than its 2.3T token counterpart. Through controlled experiments and theoretical analysis, we show that catastrophic overtraining arises from a systematic increase in the broad sensitivity of pre-trained parameters to modifications, including but not limited to fine-tuning. Our findings call for a critical reassessment of pre-training design that considers the downstream adaptability of the model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。