将工具调用与推理分离，提升大模型任务执行效率
Agent-as-Tool: A Study on the Hierarchical Decision Making with Reinforcement Learning
- 构建分层框架，由独立代理分别负责工具调用和语言推理
- 仅用180样本微调即达优异效果，Bamboogle上精确匹配率达63.2%
- 适合需要高效决策的大规模智能体系统研发人员
大语言模型（LLMs）近年来成为人工智能领域的重要突破，其自然语言理解与生成能力重塑了人机交互方式。随着基于LLM的智能体与强化学习推理模型的发展，如何在智能体框架中应用强化学习成为新研究热点。然而，现有研究面临同时决策工具调用与推理过程的难题，且推理链完全依赖未经处理的原始工具输出，其中包含冗余信息与无关符号，极大增加了模型推理负担。为此，本文提出分层框架Agent-as-Tool，将工具调用过程与推理过程解耦，使主模型专注语言推理，而工具调用由另一代理处理。实验表明，该方法仅需对180个样本进行轻微强化微调，即可在Bamboogle数据集上取得63.2%的精确匹配率与75.2%的覆盖精确匹配率，分别优于Search-R1模型4.8%和3.2%。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have emerged as one of the most significant technological advancements in artificial intelligence in recent years. Their ability to understand, generate, and reason with natural language has transformed how we interact with AI systems. With the development of LLM-based agents and reinforcement-learning-based reasoning models, the study of applying reinforcement learning in agent frameworks has become a new research focus. However, all previous studies face the challenge of deciding the tool calling process and the reasoning process simultaneously, and the chain of reasoning was solely relied on the unprocessed raw result with redundant information and symbols unrelated to the task from the tool, which impose a heavy burden on the model's capability to reason. Therefore, in our research, we proposed a hierarchical framework Agent-as-tool that detach the tool calling process and the reasoning process, which enables the model to focus on the verbally reasoning process while the tool calling process is handled by another agent. Our work had achieved comparable results with only a slight reinforcement fine-tuning on 180 samples, and had achieved exceptionally well performance in Bamboogle with 63.2% of exact match and 75.2% in cover exact match, exceeding Search-R1 by 4.8% in exact match and 3.2% in cover exact match.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。