让大模型更懂用户真实指令，通过路径规划与反馈机制提升任务完成率。
ToolPlanner: A Tool Augmented LLM for Multi Granularity Instructions with Path Planning and Feedback
- 采用两阶段强化学习框架，结合路径规划与双反馈机制优化指令执行。
- 在多粒度指令上相比顶尖模型提升匹配率26.8%、通过率20.2%、胜率5.6%。
- 适合作为智能助手的底层引擎，尤其适合需要理解模糊指令的场景。
近期，工具增强型大语言模型受到广泛关注。给定一个指令，这类模型可通过多轮与外部工具交互生成最终答案。然而，以往模型训练依赖过于详细的指令（如显式包含API名称或参数），而真实用户通常不会提及这些细节，导致模型与实际应用存在差距。此外，多数工作忽视交互过程是否真正遵循指令。为此，我们构建了名为MGToolBench的训练数据集，包含陈述级和类别级指令，更贴近真实使用场景。同时提出ToolPlanner，一种基于路径规划与双重反馈机制的两阶段强化学习框架，显著提升模型的任务完成能力与指令遵循能力。实验表明，相较于当前最优模型，ToolPlanner在匹配率、通过率和胜率上分别提升26.8%、20.2%和5.6%。人工评估验证了多粒度指令更符合用户使用习惯。数据与代码将在论文接受后公开。
原文摘要 · Abstract (English)
Recently, tool-augmented LLMs have gained increasing attention. Given an instruction, tool-augmented LLMs can interact with various external tools in multiple rounds and provide a final answer. However, previous LLMs were trained on overly detailed instructions, which included API names or parameters, while real users would not explicitly mention these API details. This leads to a gap between trained LLMs and real-world scenarios. In addition, most works ignore whether the interaction process follows the instruction. To address these issues, we constructed a training dataset called MGToolBench, which contains statement and category-level instructions to better reflect real-world scenarios. In addition, we propose ToolPlanner, a two-stage reinforcement learning framework that utilizes path planning and two feedback mechanisms to enhance the LLM's task completion and instruction-following capabilities. Experimental results show that ToolPlanner significantly improves the Match Rate, Pass Rate and Win Rate by 26.8%, 20.2%, and 5.6% compared to the SOTA model. Human evaluation verifies that the multi-granularity instructions can better align with users' usage habits. Our data and code will be released upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。