为大模型工具调用设计可验证的安全框架,防止错误结果污染推理过程。
ToolGate: Contract-Grounded and Verified Tool Execution for LLMs
- 用形式化契约控制工具调用,仅当状态满足前提条件才允许执行。
- 通过运行时验证确保结果可信,保证符号状态仅由已验证操作更新。
- 适合需要高可靠性的复杂推理任务,如自动化决策系统。
增强外部工具的大语言模型在复杂推理任务中展现出强大能力,但现有框架依赖自然语言推理来决定何时调用工具及是否提交结果,缺乏逻辑安全性和可验证性保障。本文提出ToolGate,一种前向执行框架,为大模型工具调用提供逻辑安全保障与可验证的状态演化机制。ToolGate维护一个显式的符号状态空间,以类型化键值对形式表示可信的世界信息。每个工具被形式化为霍尔风格的契约,包含前置条件和后置条件:前置条件用于判断当前状态是否满足调用条件,后置条件则决定工具结果能否通过运行时验证被提交以更新状态。该方法确保符号状态仅通过已验证的工具执行进行演化,防止无效或幻觉结果污染世界表征。实验验证表明,ToolGate显著提升了工具增强型大模型系统的可靠性与可验证性,同时在复杂多步推理任务上保持竞争力。本工作为构建更可信、可调试的集成语言模型与外部工具的AI系统奠定了基础。
原文摘要 · Abstract (English)
Large Language Models (LLMs) augmented with external tools have demonstrated remarkable capabilities in complex reasoning tasks. However, existing frameworks rely heavily on natural language reasoning to determine when tools can be invoked and whether their results should be committed, lacking formal guarantees for logical safety and verifiability. We present \textbf{ToolGate}, a forward execution framework that provides logical safety guarantees and verifiable state evolution for LLM tool calling. ToolGate maintains an explicit symbolic state space as a typed key-value mapping representing trusted world information throughout the reasoning process. Each tool is formalized as a Hoare-style contract consisting of a precondition and a postcondition, where the precondition gates tool invocation by checking whether the current state satisfies the required conditions, and the postcondition determines whether the tool's result can be committed to update the state through runtime verification. Our approach guarantees that the symbolic state evolves only through verified tool executions, preventing invalid or hallucinated results from corrupting the world representation. Experimental validation demonstrates that ToolGate significantly improves the reliability and verifiability of tool-augmented LLM systems while maintaining competitive performance on complex multi-step reasoning tasks. This work establishes a foundation for building more trustworthy and debuggable AI systems that integrate language models with external tools.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。