用多个AI代理自动优化指令数据质量，提升模型性能。
Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning
- 多智能体协作生成多样化指令数据，提升多样性与覆盖度。
- 双模型评估机制量化数据难度与质量，筛选优质样本。
- 动态迭代优化，优先使用更优模型，持续提升数据质量。
大语言模型在下游任务中的表现通常依赖于指令微调，而其效果关键取决于训练数据的质量。然而，高质量且多样化的数据收集既昂贵又耗时。为此，我们提出一种名为 Star-Agents 的新框架，通过多智能体协作与评估，实现数据集的自动化质量增强。该框架采用三阶段策略：首先，利用定制采样方法，由多个 LLM 智能体生成多样化的指令数据；其次，通过双模型方法对生成数据进行严格评估，分别衡量其难度与质量；最后，在动态优化阶段，优先使用表现更优的 LLM，持续提升整体数据质量。实验表明，该框架在 Pythia 与 LLaMA 等模型上进行指令微调时显著有效。优化后的数据集在多个基准测试中表现优异，平均性能提升达12%，特定指标如 Fermi 更提升40%，验证了其有效性。
原文摘要 · Abstract (English)
The efficacy of large language models (LLMs) on downstream tasks usually hinges on instruction tuning, which relies critically on the quality of training data. Unfortunately, collecting high-quality and diverse data is both expensive and time-consuming. To mitigate this issue, we propose a novel Star-Agents framework, which automates the enhancement of data quality across datasets through multi-agent collaboration and assessment. The framework adopts a three-pronged strategy. It initially generates diverse instruction data with multiple LLM agents through a bespoke sampling method. Subsequently, the generated data undergo a rigorous evaluation using a dual-model method that assesses both difficulty and quality. Finaly, the above process evolves in a dynamic refinement phase, where more effective LLMs are prioritized, enhancing the overall data quality. Our empirical studies, including instruction tuning experiments with models such as Pythia and LLaMA, demonstrate the effectiveness of the proposed framework. Optimized datasets have achieved substantial improvements, with an average increase of 12% and notable gains in specific metrics, such as a 40% improvement in Fermi, as evidenced by benchmarks like MT-bench, Vicuna bench, and WizardLM testset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。