arXiv:2604.12301cs.DCcs.AI2026-04被引 1

用七种方法减少代码生成中云大模型的调用次数,最多省79%成本。

Local-Splitter: A Measurement Study of Seven Tactics for Reducing Cloud LLM Token Usage on Coding-Agent Workloads

  • 本地模型先筛选请求,再配合压缩、缓存等技术降低云端调用量。
  • 在编辑和解释类任务中,组合使用本地路由与提示压缩可省45%-79%令牌。
  • 不同任务类型适用不同优化策略,实测结果对实际部署极具参考价值。

我们系统性地研究了七种减少云大模型在代码代理任务中令牌消耗的方法:(1) 本地路由,(2) 提示压缩,(3) 语义缓存,(4) 本地草稿+云端审核,(5) 最小差异修改,(6) 结构化意图提取,(7) 批处理结合厂商提示缓存。我们在一个开源适配层中实现了全部七项,支持通过Ollama接入任意本地模型,通过OpenAI兼容接口接入任意云模型。在四种代码代理工作负载(高编辑、高解释、通用对话、高RAG)上分别评估各项策略的独立效果、成对组合及贪婪叠加组合,测量了节省的令牌数、美元成本、延迟和路由准确率。主要发现是:在编辑和解释类任务中,T1(本地路由)与T2(提示压缩)组合可实现45%-79%的云端令牌节省;在高RAG任务中,包含T4(草稿-审核)在内的完整策略组合可实现51%节省。我们观察到最优策略组合随任务类型而异,这对当前部署代码代理的从业者具有最直接的指导意义。

原文摘要 · Abstract (English)

We present a systematic measurement study of seven tactics for reducing cloud LLM token usage when a small local model can act as a triage layer in front of a frontier cloud model. The tactics are: (1) local routing, (2) prompt compression, (3) semantic caching, (4) local drafting with cloud review, (5) minimal-diff edits, (6) structured intent extraction, and (7) batching with vendor prompt caching. We implement all seven in an open-source shim that speaks both MCP and the OpenAI-compatible HTTP surface, supporting any local model via Ollama and any cloud model via an OpenAI-compatible endpoint. We evaluate each tactic individually, in pairs, and in a greedy-additive subset across four coding-agent workload classes (edit-heavy, explanation-heavy, general chat, RAG-heavy). We measure tokens saved, dollar cost, latency, and routing accuracy. Our headline finding is that T1 (local routing) combined with T2 (prompt compression) achieves 45-79% cloud token savings on edit-heavy and explanation-heavy workloads, while on RAG-heavy workloads the full tactic set including T4 (draft-review) achieves 51% savings. We observe that the optimal tactic subset is workload-dependent, which we believe is the most actionable finding for practitioners deploying coding agents today.

大模型优化代码生成成本控制本地推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。