用测试算力感知提前停止训练,大幅降低训练成本
FLOP-Efficient Training: Early Stopping Based on Test-Time Compute Awareness
- 根据测试时计算量动态选择训练中途的检查点和推理配置
- 实验显示训练浮点运算量减少最高达92%,准确率不降反升
- 适合追求高效训练与快速迭代的模型研发团队
扩大训练计算量(以浮点运算次数衡量)长期以来被证明能提升大语言模型的准确性,但训练过程仍十分耗资源。先前研究指出,增加测试时计算量(如通过迭代采样)可使较小模型在更低总成本下媲美甚至超越更大模型。本文提出测试时计算感知训练方法,使一个中间检查点与对应的测试时计算配置组合,即可达到或超过全量训练模型的准确率,同时显著减少训练浮点运算量。基于此,我们设计了一种联合选择检查点与测试时计算配置的早停算法,以最小化训练计算量而不牺牲准确性。为实现实际应用,我们开发了高效的测试时计算评估方法,避免穷举搜索,并形式化了盈亏平衡边界,用于判断增加推理计算是否可抵消训练计算的减少。实验表明,在保持甚至显著提升准确率的同时,训练浮点运算量最多可减少92%。这些结果为模型开发中训练与推理计算的权衡提供了新视角,支持更快的部署周期与更频繁的模型更新。代码将公开发布。
原文摘要 · Abstract (English)
Scaling training compute, measured in FLOPs, has long been shown to improve the accuracy of large language models, yet training remains resource-intensive. Prior work shows that increasing test-time compute (TTC)-for example through iterative sampling-can allow smaller models to rival or surpass much larger ones at lower overall cost. We introduce TTC-aware training, where an intermediate checkpoint and a corresponding TTC configuration can together match or exceed the accuracy of a fully trained model while requiring substantially fewer training FLOPs. Building on this insight, we propose an early stopping algorithm that jointly selects a checkpoint and TTC configuration to minimize training compute without sacrificing accuracy. To make this practical, we develop an efficient TTC evaluation method that avoids exhaustive search, and we formalize a break-even bound that identifies when increased inference compute compensates for reduced training compute. Experiments demonstrate up to 92\% reductions in training FLOPs while maintaining and sometimes remarkably improving accuracy. These results highlight a new perspective for balancing training and inference compute in model development, enabling faster deployment cycles and more frequent model refreshes. Codes will be publicly released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。