让大模型智能识别相似指令并复用答案，提升响应速度。
Generative Caching for Structurally Similar Prompts and Responses
- 通过生成式缓存识别结构相似的指令，自动匹配可复用的回答模式。
- 在重复任务中实现83%缓存命中率，错误命中极少。
- 适合自动化工作流、智能代理等需频繁执行相似任务的场景。
大型语言模型（LLMs）正被广泛用于各类场景中的规划、推理与任务执行。在可重复的工作流和智能体设置中，提示词常以微小变化的形式重复出现，但结构保持一致。这为缓存提供了机会。然而，精确匹配无法处理结构相似的提示，而语义缓存可能因忽略关键差异导致错误响应。为此，我们提出 extit{Generative Caching}，一种能生成对变化敏感响应的生成式缓存机制。该方法识别相似提示结构中的可复用响应模式，并为新请求生成定制化输出。实验表明，该方法在无提示重复的数据集上仅有极少错误命中，缓存命中率达83%。在智能体工作流中，相比标准提示匹配，其缓存命中率提升约20%，端到端执行延迟降低约34%。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly being used to plan, reason, and execute tasks across diverse scenarios. In use cases like repeatable workflows and agentic settings, prompts are often reused with minor variations while having a similar structure for recurring tasks. This opens up opportunities for caching. However, exact prompt matching fails on such structurally similar prompts, while semantic caching may produce incorrect responses by ignoring critical differences. To address this, we introduce \ourmethod{}, a generative cache that produces variation-aware responses for structurally similar prompts. \ourmethod{} identifies reusable response patterns across similar prompt structures and synthesizes customized outputs for new requests. We show that \ourmethod{} achieves 83\% cache hit rate, while having minimal incorrect hits on datasets without prompt repetition. In agentic workflows, it improves cache hit rate by $\sim$20\% and reduces end-to-end execution latency by $\sim$34\% compared to standard prompt matching.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。