在微调时重放通用数据，能显著提升目标领域模型性能。
Replaying pre-training data improves fine-tuning
- 微调时定期重放预训练的通用文本数据
- 目标数据效率提升最高达2.06倍，8B模型任务成功率提高4.5%
- 尤其适合目标数据少、需避免遗忘的场景
为获得特定领域（如数学）的语言模型，当前方法是在大量通用网络文本上预训练，再在有限的目标数据上微调。通常仅混合少量通用数据以防止灾难性遗忘。我们意外发现，在微调过程中重放通用数据反而能提升对较不相关目标任务的表现。在控制实验中（400万目标词、40亿总词、1.5亿参数模型），通用数据重放使微调阶段目标数据效率提升最高达1.87倍，中期训练阶段达2.06倍。分析显示，当预训练阶段目标数据较少时，重放效果更明显。实际应用中，对80亿参数模型进行微调，成功提升代理式网页导航成功率4.5%，巴斯克语问答准确率2%。
原文摘要 · Abstract (English)
To obtain a language model for a target domain (e.g. math), the current paradigm is to pre-train on a vast amount of generic web text and then fine-tune on the relatively limited amount of target data. Typically, generic data is only mixed in during fine-tuning to prevent catastrophic forgetting of the generic domain. We surprisingly find that replaying the generic data during fine-tuning can actually improve performance on the (less related) target task. Concretely, in a controlled pre-training environment with 4M target tokens, 4B total tokens, and 150M parameter models, generic replay increases target data efficiency by up to $1.87\times$ for fine-tuning and $2.06\times$ for mid-training. We further analyze data schedules that introduce target data during pre-training and find that replay helps more when there is less target data present in pre-training. We demonstrate the success of replay in practice for fine-tuning 8B parameter models, improving agentic web navigation success by $4.5\%$ and Basque question-answering accuracy by $2\%$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。