用大模型自动优化工具调用文档,提升工业级工具使用的准确性和稳定性。
Verification-Guided Context Optimization for Tool Calling via Hierarchical LLMs-as-Editors
- 构建分级编辑框架,让大模型充当编辑者,自动修正工具文档与上下文的不匹配。
- 在真实失败案例上训练,使编辑后工具调用准确率显著提升,泛化能力更强。
- 适合需要大规模工具调用的工业应用,尤其适用于多工具重叠场景。
工具调用使大语言模型(LLMs)可通过调用外部工具与环境交互,从而突破预训练局限。然而,工具使用效果高度依赖文档与知识库上下文的质量。这些材料通常面向人类编写,与大模型的理解方式存在偏差。在工业场景中,数百个功能重叠的工具进一步加剧了可扩展性、差异性和歧义性问题。本文提出验证引导的上下文优化(VGCO)框架,利用大模型作为编辑者,自动优化工具相关文档与知识库上下文。该框架分为两阶段:第一阶段评估收集真实失败案例,识别工具与其上下文间的不匹配;第二阶段通过结构感知的上下文优化进行分层编辑,基于离线学习完成。本框架的创新体现在三方面:一是采用自然融入工具调用流程的分层结构;二是具备状态感知、动作特定与验证引导特性,有效约束搜索空间,实现高效精准优化;三是支持低成本子任务专业化,可通过提示工程大编辑模型或微调小编辑模型实现。不同于以往强调多轮推理的研究,VGCO聚焦单轮、大规模工具调用问题,在多种大模型上实现了准确率、鲁棒性与泛化性的显著提升。
原文摘要 · Abstract (English)
Tool calling enables large language models (LLMs) to interact with external environments through tool invocation, providing a practical way to overcome the limitations of pretraining. However, the effectiveness of tool use depends heavily on the quality of the associated documentation and knowledge base context. These materials are usually written for human users and are often misaligned with how LLMs interpret information. This problem is even more pronounced in industrial settings, where hundreds of tools with overlapping functionality create challenges in scalability, variability, and ambiguity. We propose Verification-Guided Context Optimization (VGCO), a framework that uses LLMs as editors to automatically refine tool-related documentation and knowledge base context. VGCO works in two stages. First, Evaluation collects real-world failure cases and identifies mismatches between tools and their context. Second, Optimization performs hierarchical editing through offline learning with structure-aware, in-context optimization. The novelty of our LLM editors has three main aspects. First, they use a hierarchical structure that naturally integrates into the tool-calling workflow. Second, they are state-aware, action-specific, and verification-guided, which constrains the search space and enables efficient, targeted improvements. Third, they enable cost-efficient sub-task specialization, either by prompt engineering large editor models or by post-training smaller editor models. Unlike prior work that emphasizes multi-turn reasoning, VGCO focuses on the single-turn, large-scale tool-calling problem and achieves significant improvements in accuracy, robustness, and generalization across LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。