arXiv:2605.30537cs.LG2026-05被引 1

研究数据筛选对大模型长期训练的影响,发现短期最优选择可能损害后续学习。

The Long-Term Effects of Data Selection in LLM Fine-Tuning

论文配图:The Long-Term Effects of Data Selection in LLM Fine-Tuning
图 1 · 摘自论文原文
  • 提出多阶段评估框架,考察数据选择对模型未来适应能力的影响。
  • 发现短期表现好的筛选方法会降低后续学习速度并加剧遗忘现象。
  • 提出长视野筛选目标,兼顾当前效率与长期适应性,适合长期训练场景。

数据筛选被广泛用于降低大语言模型(LLM)微调成本,现有方法多基于当前效用、多样性、质量或影响力优先选择样本。本文关注一个新问题:当微调分多个阶段进行时,当前看似最优的筛选策略是否会影响模型未来的适应能力?我们引入长周期视角下的数据筛选评估框架,不仅衡量即时任务表现,还评估未来适应速度、遗忘程度、能力失衡及分布外鲁棒性。在统一多阶段协议下,对比随机、基于损失、基于梯度、基于多样性、基于质量及效用-多样性等典型筛选方法。通过受控实验发现,短期最优的筛选策略可能出现排名反转:提升当前阶段表现的同时减缓后续学习并增加遗忘。我们将此行为定义为“短视筛选”(myopic selection),并通过局部分析揭示其成因,并提出一种诊断性长视野感知筛选(LHAS)目标,将即时效用与覆盖度、未来代理迁移性和抗集中性项相结合。研究主张,数据筛选应被视为塑造模型学习轨迹的训练干预,而非仅是局部数据效率机制。

原文摘要 · Abstract (English)

Data selection is increasingly used to reduce the cost of large language model (LLM) fine-tuning, with recent methods prioritizing samples by current utility, diversity, quality, or influence. This paper studies a different question: when fine-tuning occurs over multiple stages, can selection strategies that look optimal now make the model less adaptable later? We introduce a long-horizon view of LLM data selection in which a selector is evaluated not only by immediate task performance, but also by future adaptation speed, forgetting, capability imbalance, and out-of-distribution robustness. We compare representative random, loss-based, gradient-based, diversity-based, quality-based, and utility-diversity selection families under a unified multi-stage protocol. Through controlled experiments designed to instantiate this protocol, we show how short-term selectors can exhibit rank reversal: they improve the current stage while slowing subsequent learning and increasing forgetting. We formalize this behavior as \emph{myopic selection}, provide a simple local analysis of why it can occur, and propose a diagnostic Long-Horizon Aware Selection (LHAS) objective that augments immediate utility with coverage, future-proxy transfer, and anti-concentration terms. The study argues that data selection should be evaluated as a training intervention that shapes the model's learning trajectory, rather than only as a local data-efficiency mechanism.

大模型微调数据筛选长期适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。