用大模型自动补全工具文档,显著提升工具检索效果。
Tools are under-documented: Simple Document Expansion Boosts Tool Retrieval
- 通过大模型生成并验证工具文档,构建大规模增强语料库。
- 新模型Tool-Embed和Tool-Rank在两个基准上达最新水平。
- 适合研究工具使用与检索的开发者及研究人员。
大型语言模型(LLMs)在工具使用方面表现出色,但工具检索受限于文档不完整和格式不一。为此,我们提出Tool-DE,一个新基准与框架,通过结构化字段系统性地扩充工具文档,以提升检索效率,并设计了两款专用模型:Tool-Embed(稠密检索器)和Tool-Rank(基于LLM的重排序器)。我们构建了一条可扩展的文档扩充流水线,利用开源与闭源大模型低成本生成、验证并优化工具资料,产出50,000条用于嵌入式检索器的语料,以及200,000条用于重排序器的语料。在ToolRet和Tool-DE上的实验表明,文档扩充显著提升了检索性能,Tool-Embed与Tool-Rank在两个基准上均达到新最优结果。我们进一步分析了各字段对检索效果的贡献,以及文档扩充对训练与评估的影响。整体发现表明,大模型驱动的文档扩充具有潜力但也存在局限,而Tool-DE连同所提模型,为未来工具检索研究奠定了基础。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have recently demonstrated strong capabilities in tool use, yet progress in tool retrieval remains hindered by incomplete and heterogeneous tool documentation. To address this challenge, we introduce Tool-DE, a new benchmark and framework that systematically enriches tool documentation with structured fields to enable more effective tool retrieval, together with two dedicated models, Tool-Embed and Tool-Rank. We design a scalable document expansion pipeline that leverages both open- and closed-source LLMs to generate, validate, and refine enriched tool profiles at low cost, producing large-scale corpora with 50k instances for embedding-based retrievers and 200k for rerankers. On top of this data, we develop two models specifically tailored for tool retrieval: Tool-Embed, a dense retriever, and Tool-Rank, an LLM-based reranker. Extensive experiments on ToolRet and Tool-DE demonstrate that document expansion substantially improves retrieval performance, with Tool-Embed and Tool-Rank achieving new state-of-the-art results on both benchmarks. We further analyze the contribution of individual fields to retrieval effectiveness, as well as the broader impact of document expansion on both training and evaluation. Overall, our findings highlight both the promise and limitations of LLM-driven document expansion, positioning Tool-DE, along with the proposed Tool-Embed and Tool-Rank, as a foundation for future research in tool retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。