用桥梁模型将模糊指令转为具体指令,提升工具检索准确率。
Tool Retrieval Bridge: Aligning Vague Instructions with Retriever Preferences via Bridge Model
- 引入桥梁模型重写模糊指令,缩小与检索器偏好差距。
- 在真实场景下使BM25检索器的NDCG提升111.51%(9.73→19.59)。
- 适用于需要精准工具调用的现实应用,如智能助手、自动化系统。
工具学习已成为大语言模型应对现实挑战的有前景范式。由于工具数量庞大且更新频繁,工具检索以选出目标工具子集至关重要。然而,现有方法多基于包含详细指令(如特定API名称和参数)的学术基准,而真实指令更模糊。这种差异阻碍了实际应用中的工具检索。本文构建新基准VGToolBench模拟人类模糊指令,发现模糊指令确实损害检索性能。为此,提出简单有效的工具检索桥梁(TRB)方法:通过桥接模型将模糊指令重写为更具体的表达,缓解其与检索器偏好之间的差距。在多种常用检索设置下开展实验,结果表明TRB能有效降低模糊性,并在所有基线检索器上实现一致显著提升。例如,借助TRB,BM25的平均NDCG从9.73提升至19.59,相对改进达111.51%。代码与模型已公开于https://github.com/kfchenhn/TRB。
原文摘要 · Abstract (English)
Tool learning has emerged as a promising paradigm for large language models (LLMs) to address real-world challenges. Due to the extensive and irregularly updated number of tools, tool retrieval for selecting the desired tool subset is essential. However, current tool retrieval methods are usually based on academic benchmarks containing overly detailed instructions (e.g., specific API names and parameters), while real-world instructions are more vague. Such a discrepancy would hinder the tool retrieval in real-world applications. In this paper, we first construct a new benchmark, VGToolBench, to simulate human vague instructions. Based on this, we conduct a series of preliminary analyses and find that vague instructions indeed damage the performance of tool retrieval. To this end, we propose a simple-yet-effective Tool Retrieval Bridge (TRB) approach to boost the performance of tool retrieval for vague instructions. The principle of TRB is to introduce a bridge model to rewrite the vague instructions into more specific ones and alleviate the gap between vague instructions and retriever preferences.We conduct extensive experiments under multiple commonly used retrieval settings, and the results show that TRB effectively mitigates the ambiguity of vague instructions while delivering consistent and substantial improvements across all baseline retrievers. For example, with the help of TRB, BM25 achieves a relative improvement of up to 111.51%, i.e., increasing the average NDCG score from 9.73 to 19.59. The source code and models are publicly available at https://github.com/kfchenhn/TRB.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。