用代码生成提升大模型调用工具的效率与准确性
CodeTool: Enhancing Programmatic Tool Invocation of LLMs via Process Supervision
- 通过逐步生成代码并实时反馈，引导大模型高效调用工具
- 在StableToolBench和RestBench-TMDB上显著优于现有方法
- 适合需要精确工具调用与可验证推理的复杂任务场景
工具调用显著提升了大语言模型（LLMs）的能力，但在复杂任务中仍面临挑战。当前方法如指令增强推理和监督微调常导致过长的推理路径，且难以验证中间步骤的正确性。本文提出CodeTool框架，通过分步代码生成，利用代码的简洁性和易验证性改进大模型工具调用。该框架引入两种过程奖励：即时奖励（On-the-spot Reward），对每次工具调用的准确性提供即时反馈；潜在奖励（Latent Reward），评估每一步对任务完成的贡献度。通过最大化各步骤的累积奖励，引导模型走向高效准确的推理路径。在StableToolBench和RestBench-TMDB上的大量实验表明，CodeTool优于现有方法。
原文摘要 · Abstract (English)
Tool invocation significantly enhances the capabilities of Large Language Models (LLMs), yet challenges persist, particularly in complex task scenarios. Current methods, such as instruction-enhanced reasoning and supervised fine-tuning, often result in unnecessarily long reasoning paths and face difficulties in verifying the correctness of intermediate steps. In this paper, we propose CodeTool, a novel framework for stepwise code generation that improves LLM tool invocation by leveraging the concise and easily verifiable nature of code. CodeTool incorporates two distinct process rewards: the On-the-spot Reward, which provides immediate feedback on the accuracy of each tool invocation, and the Latent Reward, which assesses the contribution of each step toward overall task completion. By maximizing the cumulative reward of the On-the-spot and Latend Rewards at each step, LLMs are guided to follow efficient and accurate reasoning paths. Extensive experiments on StableToolBench and RestBench-TMDB demonstrate the superiority of CodeTool over existing approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。