统一动态数据训练框架,让大模型训练更高效可复现。
DataFlex: A Unified Framework for Data-Centric Dynamic Training of Large Language Models
- 整合数据选择、混合调整、重加权三种动态优化方式
- 在MMLU上比静态训练提升性能,且支持超大规模训练
- 兼容主流训练流程,适合研究者快速实验新数据策略
数据中心训练通过优化模型参数及训练数据的选择、组合与权重,成为提升大语言模型的有前景方向。然而,现有数据选择、数据混合优化和重加权方法常分散在不同代码库中,接口不一致,阻碍了可复现性、公平比较与实际集成。本文提出DataFlex,一个基于LLaMA-Factory的统一数据中心动态训练框架。DataFlex支持样本选择、领域混合调整和样本重加权三大动态优化范式,完全兼容原始训练流程。其提供可扩展的训练器抽象与模块化组件,可无缝替换标准LLM训练,并统一关键模型相关操作,如嵌入提取、推理与梯度计算,支持大规模设置(含DeepSpeed ZeRO-3)。我们在多个数据中心方法上进行综合实验:动态数据选择在MMLU上持续优于静态全数据训练,适用于Mistral-7B与Llama-3.2-3B;在Qwen2.5-1.5B上预训练于SlimPajama时,DoReMi与ODM在6B与30B token规模下均提升MMLU准确率与语料级困惑度;此外,DataFlex在运行时间上也优于原始实现。结果表明,DataFlex为数据中心动态训练提供了高效、可靠且可复现的基础设施。
原文摘要 · Abstract (English)
Data-centric training has emerged as a promising direction for improving large language models (LLMs) by optimizing not only model parameters but also the selection, composition, and weighting of training data during optimization. However, existing approaches to data selection, data mixture optimization, and data reweighting are often developed in isolated codebases with inconsistent interfaces, hindering reproducibility, fair comparison, and practical integration. In this paper, we present DataFlex, a unified data-centric dynamic training framework built upon LLaMA-Factory. DataFlex supports three major paradigms of dynamic data optimization: sample selection, domain mixture adjustment, and sample reweighting, while remaining fully compatible with the original training workflow. It provides extensible trainer abstractions and modular components, enabling a drop-in replacement for standard LLM training, and unifies key model-dependent operations such as embedding extraction, inference, and gradient computation, with support for large-scale settings including DeepSpeed ZeRO-3. We conduct comprehensive experiments across multiple data-centric methods. Dynamic data selection consistently outperforms static full-data training on MMLU across both Mistral-7B and Llama-3.2-3B. For data mixture, DoReMi and ODM improve both MMLU accuracy and corpus-level perplexity over default proportions when pretraining Qwen2.5-1.5B on SlimPajama at 6B and 30B token scales. DataFlex also achieves consistent runtime improvements over original implementations. These results demonstrate that DataFlex provides an effective, efficient, and reproducible infrastructure for data-centric dynamic training of LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。