根据样本间动态影响关系优化训练顺序，提升大模型训练效率
D$^3$: Dynamic Directional Graph-Constrained Data Scheduling for LLM Training
- 构建动态影响图，用损失关联表示样本间方向性影响
- 在预训练和微调阶段均优于现有调度方法
- 适合追求训练效率的LLM研究者和工程师
训练数据在大语言模型优化中起核心作用，促使大量研究聚焦于数据调度策略。现有方法多关注整体数据分布调整，却忽视了训练过程中样本间的相互作用。本文指出，真实数据样本常存在方向性影响，训练顺序至关重要。我们提出D³框架，将训练单元间的复杂交互建模为动态影响图，边代表基于损失的依赖关系，并在此图上求解约束优化问题，以生成尊重训练过程中信息流演变的训练序列。该方法理论严谨，在预训练与后训练阶段均持续优于现有调度方法。为保证可扩展性，D³还设计了高效近似算法，额外计算开销可控。代码已开源。
原文摘要 · Abstract (English)
Training data plays a central role in large language models (LLMs) optimization, motivating extensive research on data scheduling strategies. Most existing approaches concentrate on adjusting the overall data distribution but neglect the underlying interactions between samples during training. However, we argue that such interactions cannot be overlooked, as real-world data samples frequently exhibit directional influences on each other, making the training order crucial. Intuitively, we can prioritize train-units with greater influence to improves learning efficiency. In this work, we propose $D^3$, a Dynamic Directional graph-constrained Data scheduling framework. $D^3$ formulates the complex interactions among train-units as a dynamic influence graph, where edges represent loss-based dependencies. It then solves a constrained optimization problem over this graph to derive the training order, which ensures that the data sequence respects the evolving information flow throughout training. Our approach is theoretically motivated and yields consistent improvements over existing data scheduling methods across both pre-training and post-training phases. Furthermore, for scalability, $D^3$ also employs an efficient approximation algorithm that keeps the additional computational overhead within a manageable range. For future research, the code is available at https://github.com/xuyj233/D3.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。