提出新方法提升大模型微调效率,让数据少但更有效。
Accelerate Scaling of LLM Finetuning via Quantifying the Coverage and Depth of Instruction Set
- 通过语义覆盖和信息深度衡量数据质量,识别高效训练样本。
- 选用少量数据训练的模型性能提升更快且更持久。
- 适合追求高效微调的大模型研究者和工程团队。
大规模监督微调(SFT)数据并不必然带来性能线性增长,凸显理解训练样本有效性的重要性。本文识别出两个决定SFT可扩展性的核心数据集属性:语义覆盖(任务领域广度)与信息深度(单个样本丰富度)。我们发现,这两个属性的简单代理指标能解释实验中验证损失方差的大部分。为此,本文进一步提出模型无关的数据选择框架——信息景观近似(ILA),联合优化上述两项因素,构建逼近大规模数据信息价值的紧凑子集。实验证明,使用ILA选出的数据进行微调,模型在多种任务和模型规模下均实现更快且持续的性能提升,这一现象称为‘加速扩展’。
原文摘要 · Abstract (English)
Scaling the amount of data used for supervied fine-tuning(SFT) does not guarantee the proportional gains in model performance, highlighting a critical need to understand what makes training samples effective. This work identifies two fundamental dataset properties that govern SFT scalability: \textbf{semantic coverage}, or the breadth of task domains, and \textbf{information depth}, or the richness of individual examples. We demonstrate that simple proxies for these properties explain the majority of validation loss variance in our experiments. In this work, we further propose the \textbf{Information Landscape Approximation (ILA)}, a model-agnostic data selection framework that jointly optimizes for these two factors. ILA constructs compact subsets that approximate the informational value of large datasets. Empirical results show that models tuned on ILA-selected data achieve faster and more sustained performance improvements across diverse tasks and model sizes compared to existing methods, a phenomenon we term \textbf{accelerated scaling}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。