通过对话式纠错训练，让大模型更准调用企业工具，减少误操作。
Disambiguation-Centric Finetuning Makes Enterprise Tool-Calling LLMs More Realistic and Less Risky
- 构建三阶段对话流程，聚焦工具歧义辨析
- 在5000条真实API对话上微调，成功率提升27至49个百分点
- 提供可复用的工业级对话数据集，适合企业智能体开发
大型语言模型被越来越多地用于调用企业API，但在面对功能相近的工具或参数不明确时常常出错。本文提出DiaFORGE（对话式有机响应生成与评估框架），一种以消歧为核心的三阶段流程：(i) 生成基于角色、多轮交互的对话，要求助手区分高度相似的工具；(ii) 在30亿至700亿参数的开源模型上，使用推理轨迹进行监督微调；(iii) 通过动态套件将模型部署于实时代理循环中，评估端到端目标完成率及传统静态指标。在动态基准DiaBENCH上，经DiaFORGE训练的模型在优化提示下，工具调用成功率比GPT-4o高27个百分点，比Claude-3.5-Sonnet高49个百分点。为推动研究，我们公开了5000条生产级企业API规范及其经过严格验证的消歧对话语料，为构建可靠的企业级工具调用智能体提供实用蓝图。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly tasked with invoking enterprise APIs, yet they routinely falter when near-duplicate tools vie for the same user intent or when required arguments are left underspecified. We introduce DiaFORGE (Dialogue Framework for Organic Response Generation & Evaluation), a disambiguation-centric, three-stage pipeline that (i) synthesizes persona-driven, multi-turn dialogues in which the assistant must distinguish among highly similar tools, (ii) performs supervised fine-tuning of open-source models with reasoning traces across 3B - 70B parameters, and (iii) evaluates real-world readiness via a dynamic suite that redeploys each model in a live agentic loop and reports end-to-end goal completion alongside conventional static metrics. On our dynamic benchmark DiaBENCH, models trained with DiaFORGE raise tool-invocation success by 27 pp over GPT-4o and by 49 pp over Claude-3.5-Sonnet, both under optimized prompting. To spur further research, we release an open corpus of 5000 production-grade enterprise API specifications paired with rigorously validated, disambiguation-focused dialogues, offering a practical blueprint for building reliable, enterprise-ready tool-calling agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。