arXiv:2504.08930cs.LG2025-04被引 5

通过细粒度资源分区,让RAG系统在不加硬件的情况下更快更稳地响应请求。

VectorLiteRAG: Latency-Aware and Fine-Grained Resource Partitioning for Efficient RAG

  • 根据搜索延迟和命中率分布,动态划分向量索引在CPU/GPU间的存储位置。
  • 在不同规模模型与数据库下,请求吞吐量最高提升1.5倍,且满足延迟要求。
  • 适合需要低延迟、高并发的生产级RAG部署场景,无需额外算力。

检索增强生成(RAG)系统结合向量相似性搜索与大语言模型(LLMs),以提供准确、上下文相关的回答。然而,将向量检索器与LLM共置于共享GPU基础设施上会带来显著挑战:向量搜索对内存和I/O要求高,而LLM推理则需高吞吐与低延迟。简单的资源共享常导致性能严重下降,尤其在高请求负载或大索引尺寸下。我们提出VectorLiteRAG,一种部署友好型RAG系统,在不增加硬件资源的前提下实现符合延迟要求的推理。该系统基于详细的性能建模与访问模式分析,引入细粒度的GPU资源分配机制。通过估计搜索延迟与查询命中率分布,识别最优的索引跨CPU/GPU层级的分区点,以最小化竞争并最大化吞吐量。评估显示,相较于基线与现有先进方案,VectorLiteRAG在所有测试配置中均持续扩展了符合SLO的请求速率范围,涵盖小/大模型及小/大向量数据库。最佳情况下,其可将可达的SLO吞吐量提升至1.5倍,且不牺牲生成质量。

原文摘要 · Abstract (English)

Retrieval-Augmented Generation (RAG) systems combine vector similarity search with large language models (LLMs) to deliver accurate, context-aware responses. However, co-locating the vector retriever and the LLM on shared GPU infrastructure introduces significant challenges: vector search is memory and I/O intensive, while LLM inference demands high throughput and low latency. Naive resource sharing often leads to severe performance degradation, particularly under high request load or large index sizes. We present VectorLiteRAG, a deployment-friendly RAG system that achieves latency-compliant inference without requiring additional hardware resources. VectorLiteRAG introduces a fine-grained GPU resource allocation mechanism based on detailed performance modeling and access pattern analysis. By estimating search latency and query hit rate distributions, it identifies an optimal index partitioning point across CPU and GPU tiers to minimize contention and maximize throughput. Our evaluations show that VectorLiteRAG consistently expands the SLO compliant request rate range across all tested configurations, including both small and large LLMs, and small and large vector databases compared to naive baselines and state of the art alternatives. In the best case, VectorLiteRAG improves the attainable SLO throughput by up to 1.5 times without compromising generation quality or requiring additional compute resources.

RAG资源调度低延迟向量搜索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。