用代码调用工具比传统JSON方式更高效稳定，适合真实任务场景。
The Bitter Lesson of Tool Calling
- 将工具以Python函数形式暴露，模型通过写代码调用并执行
- 14个模型中11个在BFCL v4上表现优于或持平原生JSON调用
- 支持并行调用且在长上下文下性能更稳，适合实际应用
工具使用使大语言模型成为能超越训练数据的智能体，对具备编程能力的模型而言，程序化工具调用（PTC）通过脚本链式与并行化方式取代固定格式的JSON调用。然而，现有研究尚未在真实任务条件下，对当前及过往多代模型在标准基准上系统评估程序化工具调用。本文在BFCL v4基准上，对14个语言模型进行了实证比较。在程序化调用中，工具以带类型的Python桩函数形式提供，模型通过编写代码调用，并在单次智能体回合内完成执行与结果返回。结果显示，程序化调用在14个模型中的11个上达到或超过原生JSON调用表现，GPT-5.6系列实现10.6%的性能提升；在并行扇出场景下，13个模型表现不劣于基线；在上下文旋转（context rot）条件下，基线平均下降2.3%，而程序化调用保持稳定。结果表明，程序化工具调用是一种可行且鲁棒的替代方案，其性能与模型能力演进同步。
原文摘要 · Abstract (English)
Tool use transforms LLMs into agents that act beyond their training data, and for code-capable models, programmatic tool calling extends this further by replacing rigid JSON calls with scripts that chain and parallelize naturally. However, a systematic evaluation of tools as code on an established benchmark across current and prior model generations under real-world task conditions has not been conducted. In this work, we empirically compare programmatic tool calling (PTC) to native JSON tool calling across 14 language models on BFCL v4. In the programmatic tool calling paradigm, tools are exposed as typed Python stubs that the model invokes through code, with execution and results handled in a single agent turn. Programmatic tool calling matches or exceeds native JSON tool calling in 11 of 14 models on BFCL v4, with the GPT-5.6 family achieving a 10.6% improvement over the JSON tool calling baseline. Further, it matches or outperforms baseline in 13 of 14 models under parallel fan-out, and holds stable under context rot conditions where baseline degrades 2.3% on average. Our results demonstrate that programmatic tool calling is a viable and robust alternative to JSON tool calling, with performance tracking model capability across release generations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。