让大模型学会自我纠错工具调用错误，提升对话系统可靠性。
ToolCritic: Detecting and Correcting Tool-Use Errors in Dialogue Systems
- 设计诊断框架，识别8类工具调用错误
- 在SGD数据集上提升工具调用准确率13%
- 适合需要高可靠性的智能客服与助手场景
工具增强型大语言模型（LLMs）在现实应用中日益普及，但工具使用错误仍影响其可靠性。我们提出ToolCritic，一个用于评估和改进多轮工具增强对话中LLM行为的诊断框架。ToolCritic能检测八种特定于工具调用的错误类型，如过早调用、参数不匹配和对工具输出误解，并向主LLM提供针对性反馈。主LLM基于反馈修正响应。我们系统定义了这些错误类别，并构建合成数据集训练ToolCritic。在Schema-Guided Dialogue（SGD）数据集上的实验表明，ToolCritic相较基线方法（包括零样本提示和自校正技术）将工具调用准确率提升最高达13%。这为大模型在真实对话应用中更稳健地集成外部工具迈出重要一步。
原文摘要 · Abstract (English)
Tool-augmented large language models (LLMs) are increasingly employed in real-world applications, but tool usage errors still hinder their reliability. We introduce ToolCritic, a diagnostic framework that evaluates and improves LLM behavior in multi-turn, tool-augmented dialogues. ToolCritic detects eight distinct error types specific to tool-calling (e.g., premature invocation, argument misalignment, and misinterpretation of tool outputs) and provides targeted feedback to the main LLM. The main LLM, assumed to have strong reasoning, task understanding and orchestration capabilities, then revises its response based on ToolCritic's feedback. We systematically define these error categories and construct a synthetic dataset to train ToolCritic. Experimental results on the Schema-Guided Dialogue (SGD) dataset demonstrate that ToolCritic improves tool-calling accuracy by up to 13% over baselines, including zero-shot prompting and self-correction techniques. This represents a promising step toward more robust LLM integration with external tools in real-world dialogue applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。