自动优化指令微调任务比例，用更少 token 达到更好效果
ADAPT: Learning Task Mixtures for Budget-Constrained Instruction Tuning
- 用元学习动态调整任务采样比例，避免手动设定权重
- 在 1%-10% 令牌预算下，性能接近最优静态混合
- 自动分配资源给更难、更相关的任务，适合资源受限场景
我们提出 ADAPT，一种元学习算法，能在明确的令牌预算约束下，自动学习多任务指令微调中的任务采样比例。不同于手动固定任务权重，ADAPT 维护一个任务的连续分布，并通过平滑最差情况验证目标的元梯度进行更新，形成自适应课程，将更多令牌分配给有效任务，同时防止任务坍缩。我们在三个约 10 亿参数的开源大模型（Gemma-3-1B、LLaMA-3.2-1B、Qwen-0.6B）上实现该方法，在 20 种 Natural Instructions 任务类型上，训练预算为可用监督令牌的 1%、5% 和 10%。与均匀和按规模比例混合的强基线相比，我们在 11 个跨域基准上进行评估，涵盖推理、阅读理解、代码生成和指令遵循任务。结果表明，ADAPT 在平均下游性能上达到或略微优于最佳静态混合，同时使用更少的有效训练令牌，并将预算重新分配至更难、与基准对齐的任务。
原文摘要 · Abstract (English)
We propose ADAPT, a meta-learning algorithm that \emph{learns} task sampling proportions under an explicit token budget for multi-task instruction tuning. Instead of fixing task weights by hand, \adapt{} maintains a continuous distribution over tasks and updates it via meta-gradients of a smooth worst-case validation objective, inducing an adaptive curriculum that allocates more tokens to useful tasks while avoiding collapse. We instantiate ADAPT on three $\sim$1B-parameter open-weight LLMs (Gemma-3-1B, LLaMA-3.2-1B, Qwen-0.6B), training on 20 Natural Instructions task types under budgets of $1\%$, $5\%$, and $10\%$ of the available supervised tokens, and compare against strong supervised fine-tuning baselines with uniform and size-proportional mixing. We conduct evaluations on 11 out-of-domain benchmarks spanning reasoning, reading comprehension, code generation, and instruction following, we find that ADAPT matches or slightly improves average downstream performance relative to the best static mixture, while using fewer effective training tokens and reallocating budget toward harder, benchmark-aligned tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。