mSFT动态调整多任务微调数据混合，防止过拟合，提升模型性能。
mSFT: Addressing Dataset Mixtures Overfitting Heterogeneously in Multi-task SFT
- 通过迭代搜索识别并剔除最早过拟合的数据集，保留最优检查点继续训练。
- 在10个基准上优于4个基线，在低算力下还能降低训练浮点运算量。
- 对数据规模、任务粒度不敏感，仅需调节一个新超参数，适合实际部署。
当前语言模型训练通常对所有子数据集分配相同计算预算进行多任务监督微调（SFT），但该方法本质次优：异构学习动态导致快学任务过早过拟合，慢学任务则欠拟合。为此，我们提出mSFT，一种针对多任务数据混合的迭代式过拟合感知搜索算法。mSFT在活跃数据混合上训练模型，识别并剔除最早过拟合的子数据集，并回滚至特定最优检查点继续训练。大量实验表明，mSFT在10个基准和6个基础模型上均一致超越4个基线。进一步分析证实，mSFT在不同数据集大小、任务粒度下保持稳健增益，且对其单一新超参数（计算预算）不敏感。值得注意的是，在低计算预算下，mSFT可在提升性能的同时降低训练FLOPs。最终，mSFT建立了一种实用的过拟合感知多任务SFT算法，充分释放模型在多样数据混合中的潜力。
原文摘要 · Abstract (English)
Current language model training commonly applies multi-task Supervised Fine-Tuning (SFT) using a homogeneous compute budget across all sub-datasets. This approach is fundamentally sub-optimal: heterogeneous learning dynamics cause faster-learning tasks to overfit early while slower ones remain under-fitted. To address this, we introduce mSFT, an iterative, overfitting-aware search algorithm for multi-task data mixtures. mSFT trains the model on an active mixture, identifies and excludes the earliest overfitting sub-dataset, and reverts to that specific optimal checkpoint before continuing. Extensive evaluations demonstrate that mSFT consistently outperforms 4 baselines across 10 benchmarks and 6 base models. Further analysis confirms mSFT maintains robust gains across diverse dataset sizes, task granularities, and is insensitive to its single new hyperparameter (compute budget). Notably, at low compute budget, mSFT can improve performance while lowering training FLOPs. Ultimately, mSFT establishes a practical overfitting-aware algorithm for multi-task SFT that maximizes the potential of models across diverse data mixtures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。