用新指标选1%高质量数据,效果超全量训练。
Importance-Aware Data Selection for Efficient LLM Instruction Tuning
- 基于模型响应差异设计重要性度量MIWV,识别关键指令数据。
- 仅用Top 1% MIWV数据,性能超越全量数据训练。
- 适合追求高效微调的开发者,尤其资源受限场景。
指令微调对提升大语言模型性能与效率至关重要。其效果不仅取决于指令数据质量,也依赖于模型自身能力。研究表明,少量高质量数据即可达到甚至超过全量数据的微调效果。然而,现有方法多聚焦于数据质量评分,缺乏针对特定模型优化数据选择的策略。本文提出模型指令薄弱值(MIWV)作为新指标,通过分析模型在上下文学习下的响应差异,量化指令数据对模型能力提升的重要性。实验表明,仅使用按MIWV排序的前1%数据进行微调,性能即优于全量数据训练。该方法突破了传统数据质量评估局限,为高效指令微调提供了强实证支持。
原文摘要 · Abstract (English)
Instruction tuning plays a critical role in enhancing the performance and efficiency of Large Language Models (LLMs). Its success depends not only on the quality of the instruction data but also on the inherent capabilities of the LLM itself. Some studies suggest that even a small amount of high-quality data can achieve instruction fine-tuning results that are on par with, or even exceed, those from using a full-scale dataset. However, rather than focusing solely on calculating data quality scores to evaluate instruction data, there is a growing need to select high-quality data that maximally enhances the performance of instruction tuning for a given LLM. In this paper, we propose the Model Instruction Weakness Value (MIWV) as a novel metric to quantify the importance of instruction data in enhancing model's capabilities. The MIWV metric is derived from the discrepancies in the model's responses when using In-Context Learning (ICL), helping identify the most beneficial data for enhancing instruction tuning performance. Our experimental results demonstrate that selecting only the top 1\% of data based on MIWV can outperform training on the full dataset. Furthermore, this approach extends beyond existing research that focuses on data quality scoring for data selection, offering strong empirical evidence supporting the effectiveness of our proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。