让多个AI代理共享预计算的键值缓存,大幅降低重复计算成本。
Can I Buy Your KV Cache?

- 一个文档的键值缓存由出版方预计算,其他代理购买加载权跳过耗时的prefill阶段。
- 在Qwen3-4B上,复用计算成本仅为prefill的9-50倍,长度越长越划算。
- 适合大规模部署的AI代理系统,尤其适用于热门文档高频调用场景。
当前全球的AI代理在处理同一文档时反复重复相同计算:每次都要从头重新执行prefill,即大模型最耗算力的步骤,以重建相同的键值(KV)缓存。同样的答案被计算百万次。我们提出一个近乎简单的方案:只计算一次。由内容发布方预先计算文档的KV缓存,其他代理可付费加载并跳过prefill。该方法完全精确:加载预计算的KV继续推理与从头prefill结果一致(24/24贪婪生成令牌,逻辑层无差异),无精度损失。在Qwen3-4B上,复用计算成本仅为prefill的9-50倍,且随文本长度增长差距进一步扩大(prefill注意力复杂度为L²)。单次复用即可收回成本。关键在于缓存存储位置:若传输缓存,因KV几乎不可压缩,每加载一次的出站成本超过节省的prefill开销;而将缓存托管在服务端,如同生产环境提示缓存机制,可彻底消除出站开销。测算显示,向8000万代理提供一个3774词的热文档,重做prefill需约150万美元,而复用仅需约3万美元(节省49.7倍)。现有0.1倍缓存读取费率接口已提供10倍折扣给用户,仍在实测节省的50倍范围内,剩余空间为服务提供商利润:每篇热门文档可赚数百万美元。本文提出面向代理的预填缓存CDN架构,并将无损KV压缩与跨方支付系统留作开放问题。
原文摘要 · Abstract (English)
Right now, across the world, AI agents are repeating the same absurd act: to read one document, they each recompute it from scratch. Every agent re-runs prefill, the most compute-intensive step a large model takes, over identical text, only to rebuild a key-value (KV) cache identical to the one the agent before it just built. The same answer, computed a million times. We make a proposal that is almost offensively simple: compute it once. Let a publisher precompute a document's KV cache, and let every other agent buy the right to load it and skip prefill. It works, and it is token-exact: loading a precomputed KV and continuing matches prefilling from scratch (24/24 greedy tokens, and at the logits level), with no accuracy cost. On Qwen3-4B, reuse is 9-50x cheaper in compute than prefill, and the gap widens with length (prefill's attention scales with L^2), so a single reuse already pays it back. Then the part that matters: where the KV lives. Shipping it fails, because KV is nearly incompressible, so per-load egress costs more than the prefill it saves. Hosting it provider-side, exactly as production prompt-caching works, removes egress entirely. The size of the prize is set by our measured compute saving: serving one hot 3774-token document to 80M agents costs ~$1.5M to re-prefill but only ~$0.03M of reuse compute (49.7x less). The 0.1x cache-read tariff APIs charge passes a 10x discount to users while sitting inside this measured envelope, so the 10x is a floor that the measured ~50x compute saving clears, and the gap to the physical ~50x is provider margin: millions of dollars per popular document. We frame the resulting agent-native prefill CDN and leave lossless KV compression and a cross-party payment layer as the open problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。