arXiv:2410.12458cs.CL2024-10中稿 · ICML

用图模型兼顾数据质量与多样性,提升大模型微调效果

The Best of Both Worlds: Bridging Quality and Diversity in Data Selection with Bipartite Graph

  • 构建句子-词元二部图,用质量与多样性乘积优先级筛选数据
  • 在6个基准上优于9种基线,提升模型性能且更高效
  • 适合需要高质量多样训练数据的研究者和工程师

大语言模型的性能受监督微调数据的质量和多样性显著影响。现有方法常偏重一方,导致训练效果不佳。本文将数据选择建模为集合覆盖问题,提出GraphFilter方法,在句子与组成n-gram的二部图上,通过质量与多样性乘积定义优先级,迭代选择高优先级句子,移除已覆盖n-gram并重算优先级。在三个模型骨干、六个广泛使用的基准上验证,GraphFilter优于九种现有基线,在模型性能与计算效率上均有提升。进一步分析表明,设计选择促进更有效的子集选取,强调指令多样性价值,并揭示质量与多样性在不同子集规模下的交互机制。

原文摘要 · Abstract (English)

The performance of large language models (LLMs) is strongly influenced by the quality and diversity of data used during supervised fine-tuning (SFT). However, current data selection methods often prioritize one aspect over the other, resulting in suboptimal training outcomes. To address this, we formulate data selection as a set cover problem and present GraphFilter, a novel approach that balances both quality and diversity in data selection. GraphFilter models the dataset as a bipartite graph connecting sentences to their constituent n-grams, then employs a priority function that combines quality and diversity metrics multiplicatively. GraphFilter iteratively selects sentences with the highest priority, removes covered n-grams from the bipartite graph, and recomputes priorities to reflect the changing data landscape. We validate GraphFilter using three model backbones across six widely-used benchmarks, demonstrating that it outperforms nine existing baselines in both model performance and computational efficiency. Further analysis shows that our design choices lead to more effective subset selection, underscores the value of instruction diversity, and provides insights into how quality and diversity interact with different subset sizes.

数据筛选大模型图模型多样性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。