为大模型精准推荐任务匹配的工具集,提升复杂问题求解效率。
Task-Aligned Tool Recommendation for Large Language Models
- 基于历史使用数据动态调整工具集规模,避免冗余
- 提出多视角工具添加机制,实现高精度工具推荐
- 适用于需要外部工具增强的大模型应用开发
通过引入外部工具,大语言模型(LLMs)解决复杂问题的能力得到显著提升。然而,由于外部工具数量庞大,同时在提示中包含所有可用工具仍不现实。因此,必须为特定任务提供精确且适量的工具集合。现有工具检索方法主要关注优化工具排序列表,并直接选取固定数量的前几名工具作为工具集,但这种方法难以在执行前为不同任务提供最优工具组合,导致工具数量不匹配,出现冗余或不适用工具,影响模型对相关工具的快速获取。本文针对该问题提出任务对齐的工具推荐框架,定义了工具推荐问题并提出精度驱动的工具推荐(PTR)方法。该方法首先利用历史工具包使用数据生成初始简洁工具集,再通过工具匹配动态调整工具集,最终采用多视角策略添加工具。此外,本文构建新数据集RecTools和评估指标TRACC,用于衡量工具推荐效果。在两个公开基准和RecTools数据集上的实验表明,所提方法在准确性上表现优异。
原文摘要 · Abstract (English)
By augmenting Large Language Models (LLMs) with external tools, their capacity to solve complex problems has been significantly enhanced. However, despite ongoing advancements in the parsing capabilities of LLMs, incorporating all available tools simultaneously in the prompt remains impractical due to the vast number of external tools. Consequently, it is essential to provide LLMs with a precise set of tools tailored to the specific task, considering both quantity and quality. Current tool retrieval methods primarily focus on refining the ranking list of tools and directly packaging a fixed number of top-ranked tools as the tool set. However, these approaches often fail to equip LLMs with the optimal set of tools prior to execution, since the optimal number of tools for different tasks could be different, resulting in inefficiencies such as redundant or unsuitable tools, which impede immediate access to the most relevant tools. This paper addresses the challenge of recommending precise toolsets for LLMs. We introduce the problem of tool recommendation, define its scope, and propose a novel Precision-driven Tool Recommendation (PTR) approach. PTR captures an initial, concise set of tools by leveraging historical tool bundle usage and dynamically adjusts the tool set by performing tool matching, culminating in a multi-view-based tool addition. Additionally, we present a new dataset, RecTools, and a metric, TRACC, designed to evaluate the effectiveness of tool recommendation for LLMs. We further validate our design choices through comprehensive experiments, demonstrating promising accuracy across two open benchmarks and our RecTools dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。