将大模型长文本推理中的注意力计算与缓存管理整合为向量数据库，提升效率与生成质量。
AlayaDB: The Data Foundation for Efficient and Effective Long-context LLM Inference
- 将注意力计算和缓存分离为向量数据库的查询处理流程
- 相比现有方案在相同SLO下节省硬件资源并提升生成质量
- 适用于需要高效长上下文推理的模型即服务场景
AlayaDB 是由 AlayaDB AI 开发的面向大语言模型（LLM）长上下文推理的先进向量数据库系统。该系统将键值缓存（KV cache）和注意力计算从 LLM 推理架构中解耦，并封装为新型向量数据库。对于模型即服务（MaaS）提供商而言，在不同服务质量目标（SLO）的各类工作负载下，AlayaDB 比现有方案（如 KV 缓存拆分、基于检索的稀疏注意力）更节省硬件资源且生成质量更高。其核心在于将 LLM 推理中的注意力计算与缓存管理抽象为查询处理过程，并通过原生查询优化器实现性能优化。本文通过三个行业合作伙伴的实际用例及在 LLM 推理基准上的广泛实验，验证了 AlayaDB 的有效性。
原文摘要 · Abstract (English)
AlayaDB is a cutting-edge vector database system natively architected for efficient and effective long-context inference for Large Language Models (LLMs) at AlayaDB AI. Specifically, it decouples the KV cache and attention computation from the LLM inference systems, and encapsulates them into a novel vector database system. For the Model as a Service providers (MaaS), AlayaDB consumes fewer hardware resources and offers higher generation quality for various workloads with different kinds of Service Level Objectives (SLOs), when comparing with the existing alternative solutions (e.g., KV cache disaggregation, retrieval-based sparse attention). The crux of AlayaDB is that it abstracts the attention computation and cache management for LLM inference into a query processing procedure, and optimizes the performance via a native query optimizer. In this work, we demonstrate the effectiveness of AlayaDB via (i) three use cases from our industry partners, and (ii) extensive experimental results on LLM inference benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。