用大模型比选样本,高效提升训练数据质量。
Add-One-In: Incremental Sample Selection for Large Language Models via a Choice-Based Greedy Paradigm
- 以对比方式评估样本贡献值,而非单个样本质量。
- 在有限预算下仅需少量选择,性能超越全量数据。
- 适合需要高效筛选高质量数据的LLM训练场景。
从海量数据中选取高质量且多样化的训练样本,对降低大型语言模型(LLMs)训练开销、提升性能至关重要。现有方法多聚焦于单个样本质量评估,难以平衡多样性与数据遍历效率。本文提出一种基于选择的增量采样框架,将关注点从个体样本质量转向其加入子集后的贡献价值。利用大模型的语言理解能力,在选择过程中评估各选项的价值。设计贪心采样流程,逐步将样本加入子集,避免对整个数据集进行穷举遍历,显著提升效率。大量实验表明,所选数据不仅性能超过全量数据,还达到近期先进方法水平,且所需选择次数更少。我们在更大规模医学数据集上验证了该方法的实际应用潜力。代码与数据见https://github.com/BIRlz/comperative_sample_selection。
原文摘要 · Abstract (English)
Selecting high-quality and diverse training samples from extensive datasets plays a crucial role in reducing training overhead and enhancing the performance of Large Language Models (LLMs). However, existing studies fall short in assessing the overall value of selected data, focusing primarily on individual quality, and struggle to strike an effective balance between ensuring diversity and minimizing data point traversals. Therefore, this paper introduces a novel choice-based sample selection framework that shifts the focus from evaluating individual sample quality to comparing the contribution value of different samples when incorporated into the subset. Thanks to the advanced language understanding capabilities of LLMs, we utilize LLMs to evaluate the value of each option during the selection process. Furthermore, we design a greedy sampling process where samples are incrementally added to the subset, thereby improving efficiency by eliminating the need for exhaustive traversal of the entire dataset with the limited budget. Extensive experiments demonstrate that selected data from our method not only surpasses the performance of the full dataset but also achieves competitive results with recent powerful studies, while requiring fewer selections. Moreover, we validate our approach on a larger medical dataset, highlighting its practical applicability in real-world applications. Our code and data are available at https://github.com/BIRlz/comperative_sample_selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。