arXiv:2605.12944cs.LGcs.CL2026-05

用智能配方搜索替代单纯选数据,提升大模型微调效果

From Instance Selection to Fixed-Pool Data Recipe Search for Supervised Fine-Tuning

论文配图:From Instance Selection to Fixed-Pool Data Recipe Search for Supervised Fine-Tuning
图 1 · 摘自论文原文
  • 将数据筛选转化为可执行的配方搜索,结合过滤、混合等操作
  • 在9万条指令池上实现比全量训练更好的推理表现
  • 适合需要高效构建高质量微调数据的算法研究者

监督微调(SFT)的数据选择通常被建模为实例排序:对每个样本打分并保留前k个。然而,有效的SFT训练集往往通过有序的整理配方生成,其中过滤、混合和去重等操作共同塑造最终数据分布。本文提出固定池数据配方搜索:给定原始指令池和一组可执行操作符,目标是在有限的全量SFT评估预算下,发现一个可执行的配方,构建高质量的子集,且不生成、重写或增强训练样本。我们提出AutoSelection,一种两层求解器,通过缓存的任务、数据和模型信号解耦固定池构建与昂贵的全量评估,利用预热探测、已实现子集状态、局部配方修改、高斯过程辅助排序及停滞触发重初始化。在9万条指令池上的实验表明,AutoSelection在三个基础模型上的分布内推理平均性能最强,优于全数据训练、随机配方搜索、随机top-k及单操作符选择器。额外的分布外图推理结果、搜索稳定性分析、结构消融实验以及1.5B至7B模型迁移验证进一步表明,配方结构的重要性超越单一选择操作符。代码见https://github.com/w253/AutoSelection。

原文摘要 · Abstract (English)

Supervised fine-tuning (SFT) data selection is commonly formulated as instance ranking: score each example and retain a top-$k$ subset. However, effective SFT training subsets are often produced through ordered curation recipes, where filtering, mixing, and deduplication operators jointly shape the final data distribution. We formulate this problem as fixed-pool data recipe search: given a raw instruction pool and a library of grounded operators, the goal is to discover an executable recipe that constructs a high-quality selected subset under a limited budget of full SFT evaluations, without generating, rewriting, or augmenting training samples. We introduce AutoSelection, a two-layer solver that decouples fixed-pool materialization based on cached task-, data-, and model-side signals from expensive full evaluation, using warmup probes, realized subset states, local recipe edits, Gaussian-process-assisted ranking, and stagnation-triggered reseeding. Experiments on a 90K instruction pool show that AutoSelection achieves the strongest in-distribution reasoning average across three base models, outperforming full-data training, random recipe search, random top-$k$, and single-operator selectors. Additional Out-of-distribution graph-reasoning results, search-stability analyses, structural ablations, and 1.5B-to-7B transfer checks further show that recipe structure matters beyond individual selection operators. Code is available at https://github.com/w253/AutoSelection.

数据筛选微调优化自动搜索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。