arXiv:2608.22708cs.AI2026-08

提出双通道路由架构,解决工具调用中提示词缓存与渐进展示的矛盾。

CacheRouter: A Dual-Path Tool Routing Architecture with Cache-Preserving Main-Model Isolation for Long-Tail Tool Discovery

  • 主模型固定可见核心工具,路由通道动态选其他工具
  • 缓存命中率90.99%~95.2%,输入成本降至基准12%以下
  • 支持运行时自动注册工具,适合长尾工具发现场景

大语言模型系统中的工具使用面临结构权衡:渐进式披露通过仅显示当前任务相关工具保持提示词简短,但提示词缓存要求请求前缀固定不变;每次工具列表变更都会使缓存失效。本文将该权衡视为请求架构问题,提出双通道路由设计,将工具选择与交付分离。主模型始终看到一组固定的小型核心工具集,其请求头部保持不变;其余工具通过独立的路由通道访问,由路由器子模型在全工具列表中搜索、选择并执行工具,返回结果。工具注册可从源码自动完成,支持运行时更新,工具集扩展无需修改主模型的请求前缀。该设计泛化了渐进披露:能力通过路由通道揭示,主模型前缀保持稳定。原型在55个功能查询和30轮对话上测试,令牌级缓存命中率分别达90.99%和95.2%,在DeepSeek定价下输入成本降至无缓存基线的约12.0%和8.0%,其中缓存命中输入令牌成本约为缓存未命中令牌的1/30。

原文摘要 · Abstract (English)

Tool use in LLM systems faces a structural trade-off. Progressive disclosure keeps the prompt small by showing only the tools relevant to the current task, while prompt caching rewards a request prefix that stays fixed across calls; every change to the visible tool list invalidates the cached prefix. This paper treats the trade-off as a problem of request architecture and proposes a dual-path routing design that assigns tool selection and tool delivery to separate channels. The main model always sees a small, fixed set of core tools, so the head of its request is unchanged across calls; all other tools are reached through an independent routing channel, in which a router sub-model searches the full tool list, selects one tool, executes it, and returns the result. Tool registration is automated from source code and supports runtime updates, so the tool set can grow without modifying the main model's request prefix. The design generalizes progressive disclosure: capabilities are disclosed through the routing channel, and the main model's prefix stays stable. A prototype implementation was exercised on 55 functional queries and a 30-turn dialogue; token-level cache hit rates reached 90.99% and 95.2%, cutting input cost to about 12.0% and 8.0% of a no-cache baseline under DeepSeek's pricing, where cache-hit input tokens cost roughly 1/30 of cache-miss tokens.

工具调用提示工程缓存优化LLM架构

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。