提升大模型工具使用能力,精准生成参数避免错误调用。
TUMS: Enhancing Tool-use Abilities of LLMs with Multi-structure Handlers
- 将工具调用从粗粒度转为参数级精细处理,提升准确性。
- 在ToolQA易难任务上分别提升19.6%和50.6%的性能。
- 适合研究工具增强型大模型与自动化任务执行的开发者。
近期,大型语言模型(LLMs)在解决各类自然语言处理任务中扮演愈发重要的角色,凭借其自然语言理解与生成能力。与外部工具结合可进一步提升其有效性,提供更精确、及时和专业的响应。然而,LLMs在非可执行操作和错误操作方面仍面临挑战,主要源于参数生成不当。当前参数生成局限于工具层面,采用粗粒度策略,未考虑不同工具的难度差异。为此,我们提出TUMS框架,通过将工具级处理转化为参数级处理,以增强LLMs的工具使用能力。该框架包含四个核心组件:(1) 意图识别器,用于识别用户意图以帮助模型更好理解任务;(2) 任务分解器,将复杂任务拆解为多个涉及工具调用的子任务;(3) 子任务处理器,配备多结构参数生成模块,确保参数准确;(4) 执行器。实证研究表明,TUMS在ToolQA的易难基准上分别实现平均19.6%和50.6%的性能提升。消融实验验证了各组件的关键贡献,为工具增强型大模型的研究提供了新视角。
原文摘要 · Abstract (English)
Recently, large language models(LLMs) have played an increasingly important role in solving a wide range of NLP tasks, leveraging their capabilities of natural language understanding and generating. Integration with external tools further enhances LLMs' effectiveness, providing more precise, timely, and specialized responses. However, LLMs still encounter difficulties with non-executable actions and improper actions, which are primarily attributed to incorrect parameters. The process of generating parameters by LLMs is confined to the tool level, employing the coarse-grained strategy without considering the different difficulties of various tools. To address this issue, we propose TUMS, a novel framework designed to enhance the tool-use capabilities of LLMs by transforming tool-level processing into parameter-level processing. Specifically, our framework consists of four key components: (1) an intent recognizer that identifies the user's intent to help LLMs better understand the task; (2) a task decomposer that breaks down complex tasks into simpler subtasks, each involving a tool call; (3) a subtask processor equipped with multi-structure handlers to generate accurate parameters; and (4) an executor. Our empirical studies have evidenced the effectiveness and efficiency of the TUMS framework with an average of 19.6\% and 50.6\% improvement separately on easy and hard benchmarks of ToolQA, meanwhile, we demonstrated the key contribution of each part with ablation experiments, offering more insights and stimulating future research on Tool-augmented LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。