SLAP通过分层采样与动态批选,用更少数据实现高效指令微调。
SLAP: Stratified Loss-based Pruning for On-Policy Data-Efficient Instruction Tuning

- 基于批次整体可学性评估,结合分层采样与距离优化提升数据多样性。
- 在多模型、多任务上仅需20%-40%数据即达全量训练性能。
- 适合追求低资源高效微调的开发者和研究者使用。
指令微调虽能提升大语言模型的专用能力,但常需大量数据和长时间训练。核心挑战在于识别有效数据并高效微调。高质量且多样化的精简数据可使模型在更低成本下实现无损性能。本文提出新型批处理感知的数据选择框架SLAP,不再评估单个样本,而是评估整个批次的可学性。SLAP通过分布感知的分层采样确保数据分布覆盖全面,并通过相对距离优化最大化批内多样性。利用海塞近似梯度信息进行动态批选择,SLAP在多个模型架构(LLaMA、ChatGLM)及多样化下游任务(多轮对话、多语言翻译、问答)上显著优于现有最先进方法。尤为突出的是,相比全量数据训练,SLAP仅需20%-40%数据即可实现更优性能,大幅降低计算成本,同时保持或提升模型能力。该结果确立了SLAP在高效大语言模型指令微调中的强大潜力。
原文摘要 · Abstract (English)
Instruction tuning has optimized the specialized capabilities of large language models (LLMs), but it often requires extensive datasets and prolonged training times. The challenge lies in developing specific capabilities by identifying useful data and efficiently fine-tuning. High-quality and diverse pruned data can help models achieve lossless performance at a lower cost. In this paper, we propose \textbf{SLAP}, a novel batch-aware data selection framework that evaluates the learnability of entire batch compositions rather than individual. SLAP ensures comprehensive data distribution coverage through distribution-aware stratified sampling while maximizing intra-batch diversity through relative distance optimization. By leveraging Hessian-approximated gradient information for dynamic batch selection, SLAP significantly outperforms existing state-of-the-art methods across multiple model architectures (LLaMA, ChatGLM) and diverse downstream tasks including multi-turn dialogue, multilingual translation, and question answering. Most notably, SLAP achieves superior performance with 20-40\% less training data compared to full dataset training, substantially reducing computational costs while maintaining or improving model capabilities. These results establish SLAP as a powerful approach for efficient and effective instruction tuning of large language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。