用多个嵌入模型组合提升大模型系统语义缓存效率
An Ensemble Embedding Approach for Improving Semantic Caching Performance in LLM-based Systems
- 融合多个嵌入模型，通过元编码器统一判断语义相似性
- 在QQP数据集上实现92%缓存命中率，非等价查询拒绝率达85%
- 适合追求高效推理的大模型系统开发者
语义缓存通过识别语义相似的查询，仅存储一次响应并复用于后续等效请求，从而提升大语言模型（LLM）系统的效率。然而，现有框架依赖单一嵌入模型表示查询，难以捕捉真实查询分布中的多样化语义关系。本文提出一种集成嵌入方法，通过训练的元编码器融合多个嵌入模型，以改进语义相似性检测。我们在Quora Question Pairs（QQP）数据集上评估，测量缓存命中率、缓存未命中率、令牌节省量和响应时间。结果表明，该方法对语义等价查询的缓存命中率达92%，同时对非等价查询的正确拒绝率保持在85%。实验显示，集成嵌入方法显著优于单模型方案，在区分语义相似与不相似查询方面表现更优，有效提升缓存性能并降低计算开销。
原文摘要 · Abstract (English)
Semantic caching enhances the efficiency of large language model (LLM) systems by identifying semantically similar queries, storing responses once, and serving them for subsequent equivalent requests. However, existing semantic caching frameworks rely on single embedding models for query representation, which limits their ability to capture the diverse semantic relationships present in real-world query distributions. This paper presents an ensemble embedding approach that combines multiple embedding models through a trained meta-encoder to improve semantic similarity detection in LLM caching systems. We evaluate our method using the Quora Question Pairs (QQP) dataset, measuring cache hit ratios, cache miss ratios, token savings, and response times. Our ensemble approach achieves a 92\% cache hit ratio for semantically equivalent queries while maintaining an 85\% accuracy in correctly rejecting non-equivalent queries as cache misses. These results demonstrate that ensemble embedding methods significantly outperform single-model approaches in distinguishing between semantically similar and dissimilar queries, leading to more effective caching performance and reduced computational overhead in LLM-based systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。