用领域嵌入和合成数据提升大模型语义缓存精度与效率
Advancing Semantic Caching for LLMs with Domain-Specific Embeddings and Synthetic Data
- 采用微调后的轻量领域嵌入模型，替代通用模型
- 单轮微调后在精度和召回率上超越主流开源与商用方案
- 自研合成数据管道解决标注数据不足问题，适合生产部署
本报告研究通过使用专门微调的嵌入模型来提升语义缓存的效果。语义缓存依赖嵌入相似性而非精确键匹配，面临精度、查询延迟与计算效率之间的平衡挑战。我们提出利用小型、领域专用的嵌入模型，并结合真实世界与合成生成的数据进行微调。实验表明，仅用一个训练周期在特定数据集上微调的紧凑嵌入模型，在精度和召回率上显著优于当前最先进的开源及专有模型。此外，我们设计了一种新型合成数据生成流程，缓解了领域标注数据稀缺的问题，进一步提升了嵌入性能。该方法在计算开销与准确性之间实现了良好平衡，为实际语义缓存应用提供了高效可行的解决方案。
原文摘要 · Abstract (English)
This report investigates enhancing semantic caching effectiveness by employing specialized, fine-tuned embedding models. Semantic caching relies on embedding similarity rather than exact key matching, presenting unique challenges in balancing precision, query latency, and computational efficiency. We propose leveraging smaller, domain-specific embedding models, fine-tuned with targeted real-world and synthetically generated datasets. Our empirical evaluations demonstrate that compact embedding models fine-tuned for just one epoch on specialized datasets significantly surpass both state-of-the-art open-source and proprietary alternatives in precision and recall. Moreover, we introduce a novel synthetic data generation pipeline for the semantic cache that mitigates the challenge of limited domain-specific annotated data, further boosting embedding performance. Our approach effectively balances computational overhead and accuracy, establishing a viable and efficient strategy for practical semantic caching implementations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。