用影响蒸馏法高效筛选训练数据，提升大模型微调速度与效果
Efficient Data Selection at Scale via Influence Distillation
- 基于二阶信息计算样本影响，为每个样本生成模型专属权重
- 在Tulu V2数据集上实现3.5倍加速，性能达或超越当前最优
- 适合追求高效微调的大模型研发人员，尤其关注算力优化场景
有效的数据选择对现代大语言模型（LLM）的高效训练至关重要。本文提出影响蒸馏（Influence Distillation），一种数学严谨的数据选择框架，利用二阶信息为训练样本分配最优权重。通过蒸馏每个样本对目标分布的影响，该方法生成模型特定权重，用于指导大模型微调以在目标领域获得优异表现。我们推导了梯度下降和Adam优化器下的最优权重。为保证可扩展性并降低计算成本，提出基于‘地标’的近似方法：仅精确计算少量‘地标’样本的影响，再高效传播至所有其他样本以确定其权重。我们在Tulu V2数据集上进行指令微调实验，覆盖GSM8k、SQuAD、MMLU等任务，测试了Llama和Qwen系列多个模型。结果表明，该方法在达到或超越当前最先进性能的同时，实现最高3.5倍的数据选择速度提升。
原文摘要 · Abstract (English)
Effective data selection is critical for efficient training of modern Large Language Models (LLMs). This paper introduces Influence Distillation, a novel, mathematically-justified framework for data selection that employs second-order information to optimally weight training samples. By distilling each sample's influence on a target distribution, our method assigns model-specific weights that are used to select training data for LLM fine-tuning, guiding it toward strong performance on the target domain. We derive these optimal weights for both Gradient Descent and Adam optimizers. To ensure scalability and reduce computational cost, we propose a $\textit{landmark-based approximation}$: influence is precisely computed for a small subset of "landmark" samples and then efficiently propagated to all other samples to determine their weights. We validate Influence Distillation by applying it to instruction tuning on the Tulu V2 dataset, targeting a range of tasks including GSM8k, SQuAD, and MMLU, across several models from the Llama and Qwen families. Experiments show that Influence Distillation matches or outperforms state-of-the-art performance while achieving up to $3.5\times$ faster selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。