让大模型更聪明地用工具，解决多步操作中的计划错误和工具误用问题。
PEARL: Plan Exploration and Adaptive Reinforcement Learning for Multihop Tool Use
- 分两阶段训练：先离线探索工具用法，再在线强化学习优化规划
- 在ToolHop数据集上达到56.5%的成功率，远超现有方法
- 适合需要复杂工具调用的智能体开发与系统集成场景
大型语言模型在使用外部工具时潜力巨大，但在复杂的多轮工具调用中仍面临严重挑战，包括规划能力弱、工具幻觉、参数生成错误以及交互鲁棒性差。为此，我们提出PEARL框架，通过增强大模型在复杂工具使用中的规划与执行能力来应对这些难题。该框架采用两阶段策略：第一阶段为离线探索，代理通过试错学习工具的有效使用模式与失败条件；第二阶段为在线强化学习，利用群组相对策略优化（GRPO）训练专用规划器，并设计了能区分规划质量的奖励函数。在ToolHop和T-Eval基准测试中，PEARL显著优于现有方法，在ToolHop上取得56.5%的新记录成功率，同时保持较低的调用错误率。本工作标志着解决大模型工具使用中复杂规划难题的重要进展，推动了更鲁棒、可靠的基于大模型智能体的发展。
原文摘要 · Abstract (English)
Large Language Models show great potential with external tools, but face significant challenges in complex, multi-turn tool invocation. They often exhibit weak planning, tool hallucination, erroneous parameter generation, and struggle with robust interaction. To tackle these issues, we present PEARL, a novel framework to enhance LLM planning and execution for sophisticated tool use. PEARL adopts a two-stage approach: an offline phase where the agent explores tools to learn valid usage patterns and failure conditions, and an online reinforcement learning phase. In the online phase, a dedicated Planner is trained via group Relative Policy Optimization (GRPO) with a carefully designed reward function that provides distinct signals for planning quality. Experiments on the ToolHop and T-Eval benchmarks show PEARL significantly outperforms existing methods, achieving a new state-of-the-art success rate of \textbf{56.5\%} on ToolHop while maintaining a low invocation error rate. Our work marks a key advance in addressing the complex planning challenges of tool use, contributing to the development of more robust and reliable LLM-based agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。