用小模型预选重要数据，大幅降低大模型微调的计算开销。
Speculative Coreset Selection for Task-Specific Fine-tuning
- 用同族小模型预估数据重要性，再在大模型上验证，精准筛选关键样本。
- 在不同删减率下，性能比现有方法最高提升54.3%，选择耗时减少70.5%。
- 低删减率（20%）下选中的数据集，微调效果甚至优于全量数据。
任务特定微调对大语言模型部署至关重要，但需要大量计算资源和时间。现有核心子集选择方法存在两大局限：高删减率下遗漏有价值样本，导致核心子集性能下降；核心子集选择过程耗时过高，需频繁微调并评估目标LLM。本文提出STAFF，一种推测式核心子集选择方法。STAFF利用与目标LLM同族的小模型高效估算数据得分，并在目标LLM上验证得分，精准识别并分配更多选择预算至重要区域，同时保持对简单区域的覆盖。我们在三个LLM和三个下游任务上评估STAFF，结果表明其性能比现有最佳方法最高提升54.3%，选择开销最高降低70.5%。此外，我们发现，在低删减率（20%）下由STAFF选出的核心子集，其微调表现甚至优于全量数据。
原文摘要 · Abstract (English)
Task-specific fine-tuning is essential for the deployment of large language models (LLMs), but it requires significant computational resources and time. Existing solutions have proposed coreset selection methods to improve data efficiency and reduce model training overhead, but they still have limitations: 1) Overlooking valuable samples at high pruning rates, which degrades the coreset's performance. 2) Requiring high time overhead during coreset selection to fine-tune and evaluate the target LLM. In this paper, we introduce STAFF, a speculative coreset selection method. STAFF leverages a small model from the same family as the target LLM to efficiently estimate data scores and then verifies the scores on the target LLM to accurately identify and allocate more selection budget to important regions while maintaining coverage of easy regions. We evaluate STAFF on three LLMs and three downstream tasks and show that STAFF improves the performance of SOTA methods by up to 54.3% and reduces selection overhead by up to 70.5% at different pruning rates. Furthermore, we observe that the coreset selected by STAFF at low pruning rates (i.e., 20%) can even obtain better fine-tuning performance than the full dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。