用指南有效性筛选高质量数据，提升大模型代理训练效率
EDGE: Efficient Data Selection for LLM Agents via Guideline Effectiveness
- 通过指南有效性指标筛选难样本，无需真实答案
- 在HotpotQA和WebShop上分别少用75%和50%数据仍更优
- 适合关注数据质量与训练效率的LLM研究者
大型语言模型（LLMs）在作为人工智能代理方面展现出卓越能力。然而，现有提升LLM代理能力的方法往往忽视数据质量，导致微调和提示工程效率低下、效果不佳。为此，我们提出EDGE方法，无需黄金答案即可识别信息量高的样本。引入指南有效性（GE）度量，通过评估人类提供的指南在多轮交互任务中的影响来筛选挑战性样本。低GE得分表明指南中缺失人类专业知识，此类样本更具信息量。通过选择低GE得分样本，可显著提升提示工程与微调的效率和效果。大量实验验证了该方法的有效性：在HotpotQA和WebShop数据集上，分别仅需75%和50%的数据即达到竞争性表现，且优于现有方法。本研究为LLM代理微调的数据质量提供了新视角。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown remarkable capabilities as AI agents. However, existing methods for enhancing LLM-agent abilities often lack a focus on data quality, leading to inefficiencies and suboptimal results in both fine-tuning and prompt engineering. To address this issue, we introduce EDGE, a novel approach for identifying informative samples without needing golden answers. We propose the Guideline Effectiveness (GE) metric, which selects challenging samples by measuring the impact of human-provided guidelines in multi-turn interaction tasks. A low GE score indicates that the human expertise required for a sample is missing from the guideline, making the sample more informative. By selecting samples with low GE scores, we can improve the efficiency and outcomes of both prompt engineering and fine-tuning processes for LLMs. Extensive experiments validate the performance of our method. Our method achieves competitive results on the HotpotQA and WebShop and datasets, requiring 75\% and 50\% less data, respectively, while outperforming existing methods. We also provide a fresh perspective on the data quality of LLM-agent fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。