用开放标签与对比评分，高效选出更优质多样的指令微调数据。
TACOS: Open Tagging and Comparative Scoring for Instruction Fine-Tuning Data Selection
- 用大模型给查询打开放标签，再去噪聚类，提升数据多样性。
- 在聚类内对比样本质量，避免单个样本评价标准不一。
- 在MT-Bench和AlpacaEval上表现领先，适合构建高质量指令数据集。
指令微调（IFT）对对齐大语言模型与人类偏好至关重要，从海量数据中选取少量代表性子集可显著提升效率与效果。然而现有方法存在两大局限：使用简单启发式导致数据多样性不足，且单样本质量评估标准不一致。为此，我们提出TACOS，融合开放标签与对比评分的指令微调数据选择方法。通过大模型为人类查询生成开放域标签，并经归一化去噪后实现高效聚类，以捕捉数据多样性；同时提出对比评分机制，在聚类内部对样本进行相对质量评估，避免单样本评价标准不一致问题。在多种数据集与大模型架构上的实验表明，TACOS显著优于现有方法，尤其在MT-Bench上表现优异，且在AlpacaEval 2.0上位列所有LLaMA2-7B基线模型第一，验证了其在指令微调数据选择中的有效性。
原文摘要 · Abstract (English)
Instruction Fine-Tuning (IFT) is crucial for aligning large language models (LLMs) with human preferences, and selecting a small yet representative subset from massive data significantly facilitates IFT in terms of both efficiency and effectiveness. Nevertheless, existing approaches suffer from two limitations: the use of simple heuristics restricts data diversity, while the singleton data quality evaluation accounts for inconsistent criteria between independent samples. To address the issues, we present TACOS, an innovative method that integrates Open Tagging and Comparative Scoring for IFT data selection. To capture data diversity, we leverage LLMs to assign open-domain tags to human queries, followed by a normalization stage to denoise the open tags and enable efficient clustering. Additionally, we suggest a comparative scoring method that allows the relative quality evaluation of samples within a cluster, avoiding inconsistent criteria seen in singleton-based evaluations. Extensive experiments across diverse datasets and LLM architectures demonstrate that TACOS outperforms existing approaches by a large margin. Notably, it achieves superior instruction-following performance on MT-Bench and ranks 1st among LLaMA2-7B-Based models on AlpacaEval 2.0, illustrating its efficacy for IFT data selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。