用最优控制理论优化语言模型预训练数据选择，提升学习效率与性能。
Data Selection via Optimal Control for Language Models
- 将数据选择建模为最优控制问题，利用庞特里亚金原理推导出最优条件。
- 在CommonCrawl数据上筛选后，模型训练加速，下游任务性能持续提升。
- 适用于大模型（~400B参数）和数据受限场景，可降低1.8倍数据需求。
本文研究从海量语料中选取高质量预训练数据以增强语言模型在下游任务中的能力。将数据选择建模为广义最优控制问题，理论上通过庞特里亚金最大值原理（PMP）求解，得到刻画最优数据选择与语言模型训练动态关系的必要条件。基于此，提出基于PMP的数据选择（PDS）框架，通过近似求解PMP条件实现最优数据筛选。实验采用PDS从CommonCrawl中选数，结果表明，经PDS筛选的语料能加速语言模型学习，并在多种模型规模下持续提升其在广泛下游任务上的表现。此外，对约400B参数、使用约10T token训练的模型，根据缩放定律外推测试损失曲线，验证了PDS的显著优势。当预训练数据有限时，PDS可将数据需求减少1.8倍，有助于缓解网络爬取语料的快速耗尽。代码、模型与数据见：https://github.com/microsoft/LMOps/tree/main/data_selection。
原文摘要 · Abstract (English)
This work investigates the selection of high-quality pre-training data from massive corpora to enhance LMs' capabilities for downstream usage. We formulate data selection as a generalized Optimal Control problem, which can be solved theoretically by Pontryagin's Maximum Principle (PMP), yielding a set of necessary conditions that characterize the relationship between optimal data selection and LM training dynamics. Based on these theoretical results, we introduce PMP-based Data Selection (PDS), a framework that approximates optimal data selection by solving the PMP conditions. In our experiments, we adopt PDS to select data from CommmonCrawl and show that the PDS-selected corpus accelerates the learning of LMs and constantly boosts their performance on a wide range of downstream tasks across various model sizes. Moreover, the benefits of PDS extend to ~400B models trained on ~10T tokens, as evidenced by the extrapolation of the test loss curves according to the Scaling Laws. PDS also improves data utilization when the pre-training data is limited, by reducing the data demand by 1.8 times, which helps mitigate the quick exhaustion of available web-crawled corpora. Our code, model, and data can be found at https://github.com/microsoft/LMOps/tree/main/data_selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。