用分层剪枝方法高效选微调数据,省时省力还更准。
HARP: Efficient Data Selection for Finetuning Large Language Models

- 构建数据层次结构,只评估代表性样本,用贝叶斯推断估算其他数据价值
- 在仅使用约1/7训练样本情况下,性能比最强基线高8.9点
- 适合大规模语言模型微调中需要快速选优数据的场景
微调数据选择需平衡提升下游任务性能与降低计算成本。无训练筛选器虽可扩展但依赖嵌入相似性等代理指标,可能与目标不匹配;基于训练的筛选器虽能通过梯度信号或Shapley值更好反映下游效用,但需多次耗时的训练-评估迭代。本文提出分层主动区域剪枝(HARP),一种高效的基于训练的筛选器,在保持下游对齐的同时大幅降低选择成本。HARP将训练池组织为节点-叶子层级结构,仅评估代表性叶子,并用经验贝叶斯后验推断未测量样本的效用。随后通过两种互补策略:保守控制冗余的HARP-C,以及奖励互补区域的HARP-E进行数据选择。理论上,在局部平滑性和有界估计误差条件下,HARP可控制选择误差并减少训练-评估开销。实验证明,HARP变体达到最优效果,相比最强基线最高提升8.9点,且仅使用约7倍更少的训练样本。
原文摘要 · Abstract (English)
Finetuning data selection requires balancing two competing goals: selecting examples that improve the downstream objective, and doing so without repeatedly finetuning models. Train-free selectors are scalable but rely on proxies such as embedding similarity or clustering, which may not match the target objective. Train-based selectors better reflect downstream utility through gradient signals, subset evaluation, or Shapley attribution, but require many costly train--evaluate iterations. We propose Hierarchical Active Region Pruning (HARP), an efficient train-based selector that preserves downstream alignment while reducing selection cost. HARP organizes the training pool into a node--leaf hierarchy, evaluates only representative leaves, and infers unmeasured utilities with empirical Bayes posteriors. It then selects data using two complementary envelopes: HARP-C, which conservatively controls redundancy, and HARP-E, which additively rewards complementary regions. We theoretically show that, under local smoothness and bounded estimation error, HARP controls selection error while reducing train--evaluate cost. We further validate that HARP variants achieve the best result and outperform the strongest baseline by up to $+8.9$ points, while using roughly $7\times$ fewer training examples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。