让大模型对话更聪明：智能缓存记住上下文，减少重复计算
ContextCache: Context-Aware Semantic Cache for Multi-Turn Queries in Large Language Models
- 用两阶段检索+注意力机制融合当前与历史对话，精准匹配上下文
- 相比现有方法，查准率和查全率均提升，缓存响应延迟降低约10倍
- 适合需要高效多轮对话的AI应用，如客服机器人、智能助手
语义缓存通过存储并复用大语言模型（LLM）的回复，显著降低计算成本并提升效率。然而，现有系统主要依赖单一查询匹配，缺乏对多轮对话上下文的感知，导致在不同对话情境中出现相似查询时产生错误缓存命中。本演示提出ContextCache，一种面向多轮对话的上下文感知语义缓存系统。该系统采用两阶段检索架构：首先基于向量对当前查询进行初步匹配，随后通过自注意力机制融合当前与历史对话表示，实现精确的上下文匹配。真实对话评估表明，ContextCache在精度和召回率上优于现有方法。此外，缓存响应的延迟约为直接调用LLM的十分之一，显著降低了大语言模型对话应用的计算开销。
原文摘要 · Abstract (English)
Semantic caching significantly reduces computational costs and improves efficiency by storing and reusing large language model (LLM) responses. However, existing systems rely primarily on matching individual queries, lacking awareness of multi-turn dialogue contexts, which leads to incorrect cache hits when similar queries appear in different conversational settings. This demonstration introduces ContextCache, a context-aware semantic caching system for multi-turn dialogues. ContextCache employs a two-stage retrieval architecture that first executes vector-based retrieval on the current query to identify potential matches and then integrates current and historical dialogue representations through self-attention mechanisms for precise contextual matching. Evaluation of real-world conversations shows that ContextCache improves precision and recall compared to existing methods. Additionally, cached responses exhibit approximately 10 times lower latency than direct LLM invocation, enabling significant computational cost reductions for LLM conversational applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。