自然语言工具让AI代理更准更快，尤其适合不擅长结构化调用的模型。
The Remarkable Effectiveness of Providing AI Agents with Natural Language Tools: A Replication Study Validating NLT Performance Across 14 Models
- 用自然语言代替结构化指令调用工具，提升准确率14.9个百分点。
- 关键错误减少93%，令牌消耗降低25.2%，在复杂任务中优势更明显。
- 特别适合生产环境，对小型或推理型模型效果显著，可靠性更高。
本研究独立复现并扩展了Johnson等人（2025）提出的自然语言工具（NLT）框架，该框架质疑大语言模型代理系统中结构化工具调用的有效性。我们在14个模型上进行了8,560次试验，涵盖前沿模型、推理模型和开源模型。结果验证了核心发现：NLT整体将工具调用准确率提升至62.3%（对比结构化调用的47.4%），关键错误减少93%（从755降至51例）。性能增益与模型能力相关：无原生工具调用能力、推理类及小型模型获益显著（+24.0pp至+43.1pp），而高度优化的前沿模型（如GPT-5、Gemini 2.5 Pro）增益较小甚至出现倒退，符合近期强化学习优化工具使用的分析。此外，NLT减少25.2%的令牌使用。在递归代理工作流中，结构化失败引发重试、备用路由与协调开销，而NLT从源头避免此类成本。本文贡献包括：首次基于开源工具独立验证NLT；揭示模型能力调节其收益（Chen et al., 2025; Zhang et al., 2025）；量化其可靠性优势（93%错误减少），这是部署中最关键的特性。NLT是结构化调用的实用替代方案，尤其适用于重视可靠性的生产系统。
原文摘要 · Abstract (English)
This study independently replicates and extends the Natural Language Tools (NLT) framework of Johnson et al.~(2025), which questions the use of structured tool calling in large language model (LLM) agentic systems. We evaluated NLT across 14 models and 8,560 trials, adding newer frontier, reasoning, and open-weight models to the original set. The results confirm the core findings and add detail. NLT improves tool-calling accuracy by 14.9 percentage points overall (62.3\% versus 47.4\% structured) and reduces critical errors by 93\% (51 versus 755 errors). The gains depend on model capability: models without native tool calling, reasoning models, and smaller models gain substantially (+24.0pp to +43.1pp), while heavily optimized frontier models (GPT-5, Gemini 2.5 Pro) show smaller or reversed advantages. This matches recent analyses of reinforcement-learning-optimized tool use (Martinez, 2025). NLT also cuts token usage by 25.2\%. The reliability and efficiency advantages compound in recursive agentic workflows, where agents chain many tool calls across sub-agents: a structured failure triggers retries, fallback routing, and coordination overhead, while NLT avoids most of that cost at the source. This work makes three contributions: (1) the first independent validation of NLT using open-source tooling, (2) evidence that model capability moderates NLT's advantages (Chen et al., 2025; Zhang et al., 2025), and (3) a measurement of NLT's reliability benefit (93\% fewer errors), its most deployment-relevant property given the known fragility of structured tool calling. NLT is a practical alternative to structured tool calling, especially for production systems that value reliability over parseability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。