通过元验证与反思学习，提升大模型用工具解决复杂问题的能力。
Advancing Tool-Augmented Large Language Models via Meta-Verification and Reflection Learning
- 构建元验证管道生成高质量工具数据集，解决工具调用幻觉问题。
- 引入探索式反思学习，使模型纠错率提升至58.9%，远超前代。
- 适用于需要高可靠工具调用与自我修正能力的AI代理系统。
赋予大语言模型有效的工具使用能力对实现AI智能体解决复杂问题至关重要。然而当前模型面临两大瓶颈：(1) 由于低质量指令数据集（如广泛存在的API调用幻觉）导致工具规划与调用不可靠；(2) 依赖静态模仿学习，工具反思能力弱（超过90%错误无法纠正）。为此，我们提出Tool-MVR，一种新型工具增强型大模型，通过两项关键创新实现全面的系统2推理。首先，我们引入多智能体元验证（MAMV），系统性地验证API、查询和推理路径，构建高质量指令数据集ToolBench-V，解决工具调用不可靠问题。其次，提出基于探索的反思学习（EXPLORE），通过动态“错误→反思→修正”机制利用工具反馈，生成反思数据集ToolBench-R，显著提升工具反思能力。最终，我们将开源模型（如Qwen-7B）在ToolBench-V与ToolBench-R上微调得到Tool-MVR。实验表明，Tool-MVR在StableToolBench上达到顶尖性能，较ToolLLM提升23.9%，较GPT-4提升15.3%，同时减少31.4%的API调用，并具备跨未见工具与场景的强泛化能力。此外，在我们提出的首个评估工具反思能力的基准RefineToolBench上，Tool-MVR错误纠正率达58.9%，显著优于ToolLLM的9.1%。
原文摘要 · Abstract (English)
Empowering large language models (LLMs) with effective tool utilization capabilities is crucial for enabling AI agents to solve complex problems. However, current models face two major limitations: (1) unreliable tool planning and invocation due to low-quality instruction datasets (e.g., widespread hallucinated API calls), and (2) weak tool reflection abilities (over 90% of errors cannot be corrected) resulting from static imitation learning. To address these critical limitations, we propose Tool-MVR, a novel Tool-Augmented LLM that achieves comprehensive System 2 reasoning through two key innovations. Specifically, we first introduce Multi-Agent Meta-Verification (MAMV), a systematic pipeline that rigorously validates APIs, queries, and reasoning trajectories to construct ToolBench-V, a new high-quality instruction dataset that addresses the limitation of unreliable tool planning and invocation. Second, we propose Exploration-based Reflection Learning (EXPLORE), which enhances tool reflection capabilities by leveraging tool feedback through a dynamic "Error -> Reflection -> Correction" learning paradigm, resulting in our reflection dataset ToolBench-R and addressing the critical weakness in tool reflection. Finally, we obtain Tool-MVR by finetuning open-source LLMs (e.g., Qwen-7B) on both ToolBench-V and ToolBench-R. Our experiments demonstrate that Tool-MVR achieves state-of-the-art performance on StableToolBench, surpassing both ToolLLM (by 23.9%) and GPT-4 (by 15.3%) while reducing API calls by 31.4%, with strong generalization capabilities across unseen tools and scenarios. Additionally, on our proposed RefineToolBench, the first benchmark specifically designed to evaluate tool reflection capabilities, Tool-MVR achieves a 58.9% error correction rate, significantly outperforming ToolLLM's 9.1%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。