Switchcraft智能选型，让工具调用更省钱不降质
Switchcraft: AI Model Router for Agentic Tool Calling
- 根据任务自动选最便宜的模型，保证正确性
- 准确率达82.9%，推理成本降低84%（每百万请求省超3600美元）
- 适合追求性价比的智能体系统开发者
能够调用外部工具的智能体AI系统功能强大但成本高昂，开发者常默认使用大模型导致推理预算浪费。现有模型路由方法针对对话生成设计，不适用于工具调用。我们提出Switchcraft，首个专为智能体工具调用优化的模型路由器。它在线运行，选择在满足正确性前提下成本最低的模型。我们在五个函数调用基准上构建评估框架，训练基于DistilBERT的分类器，并在延迟约束下部署。Switchcraft达到82.9%准确率，与最优单模型相当，同时推理成本降低84%，每百万查询节省超过3600美元。研究发现，大模型在工具任务上并非始终优于小模型，且看似便宜的模型可能因生成过多令牌而总成本更高。本工作实现了无需牺牲准确性的成本感知智能体部署。
原文摘要 · Abstract (English)
Agentic AI systems that invoke external tools are powerful but costly, leading developers to default to large models and overspend inference budgets. Model routing can mitigate this, but existing routers are designed for chat completion rather than tool use. We present Switchcraft, the first (to the best of our knowledge) model router optimized for agentic tool calling. Switchcraft operates inline, selecting the lowest-cost model subject to correctness. We construct an evaluation framework on five function-calling benchmarks and train a DistilBERT-based classifier, deployed under a latency budget. Switchcraft achieves 82.9% accuracy -- matching or exceeding the best individual model -- while reducing inference cost by 84%, saving over $3,600 per million queries. We find that larger models do not consistently outperform smaller ones on tool-use tasks, and that nominally cheaper models can incur higher total cost due to token-intensive reasoning. Our work enables cost-aware agentic AI deployment without sacrificing correctness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。