arXiv:2608.08654cs.AI2026-08

Agent的架构设计比接口方式对成本影响更大,实测差距可达28倍。

The Scaffolding Matters More Than the Interface: A Controlled Comparison of MCP and CLI Tool Use Across Seven Agent Scaffoldings, Five Language Models, and One Software Task

  • 对比MCP与CLI在七种架构中的使用成本,发现架构差异是主要影响因素。
  • 无MCP支持的架构仅用CLI完成任务,成本为有MCP架构的5.0至28倍。
  • 模型规模越小,架构差异带来的成本波动越大,最高达139倍,适合关注效率的开发者。

AI编程代理的运行成本可能更多取决于驱动它的代理架构,而非工具调用接口。我们测量了在模型上下文协议(MCP)与普通命令行接口(CLI)下工具使用的成本差异,此前公开估算相差超一个数量级,且依赖无法复现的实践报告。在七个代理架构、五种语言模型上执行同一软件任务——对私有在线git仓库进行六次操作,并通过检查仓库状态验证结果,而非依赖代理自述。主导效应来自架构:其中两个架构完全不支持MCP,全程使用CLI,其成本比支持MCP的五个架构低5.0至28倍。对于本地运行的小型270亿参数模型,成本在不同架构间波动达139倍,但所有架构均成功完成任务。原计划的成对比较结果极不稳定:13组严格配对的MCP/CLI成本比范围为0.43倍至29倍,两侧均有异常值。在失败成本上,MCP支出中12.9%未产生有效成果,而CLI仅为2.2%;但失败频率在两者间无显著差异。代理常无视分配的接口,导致非验证性比较测得的是未知混合结果。实验框架、任务、验证方法及完整数据集已开源。

原文摘要 · Abstract (English)

How much an AI coding agent costs to run can depend more on the agent scaffolding that drives it than on the interface through which it reaches its tools. We set out to measure the cost of tool use over the Model Context Protocol (MCP) against tool use over an ordinary command-line interface (CLI), a difference on which published estimates disagree by more than an order of magnitude while resting on practitioner reports that cannot be reproduced. We ran one fixed software task -- six operations against a private online git repository -- across seven agent scaffoldings and five language models, and we verified completion by inspecting the repository state rather than trusting the agent's self-report. The dominant effect was the scaffolding. Two of the seven ship no MCP support at all; they completed every run using only the CLI, which shows that MCP is unnecessary for this class of work, and they were 5.0x to 28x cheaper than the five scaffoldings that do support MCP, comparing CLI runs alone with no MCP server attached anywhere. The effect was largest for a small 27-billion-parameter model running locally, whose cost varied 139x across scaffoldings while it completed the task under all of them. The comparison we set out to make proved unstable: thirteen strictly paired MCP-to-CLI ratios span 0.43x to 29x, with outliers on both sides. The two interfaces separate on the cost of failure, where 12.9 per cent of the money spent on MCP runs bought no completed work against 2.2 per cent on CLI runs, but not on its frequency: failures were equally common in both, in the original runs and in their repetitions alike. Agents frequently ignored the interface they were assigned, so comparisons that do not verify actual behaviour measure an unknown mixture. The harness, the task, the verification and the complete dataset are released as open source.

AI编程成本优化代理架构MCP

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。