RAGO优化检索增强生成服务，提升效率并降低延迟。
RAGO: Systematic Performance Optimization for Retrieval-Augmented Generation Serving
- 提出RAGSchema统一抽象多种RAG算法，便于系统优化。
- 实测显示每芯片吞吐量提升2倍，首次响应延迟降低55%。
- 适合需要高效部署RAG系统的工程师和研究者使用。
检索增强生成（RAG）结合大语言模型与外部知识库检索，正成为可靠LLM服务的主流方法。然而，由于RAG变体迅速增多且工作负载特性差异显著，高效RAG服务仍是开放挑战。本文提出三项核心贡献：首先，引入RAGSchema，一种结构化抽象，涵盖多种RAG算法，为性能优化奠定基础；其次，分析多个具有不同RAGSchema的代表性工作负载，揭示其性能存在显著差异；第三，提出RAGO（检索增强生成优化器），一个面向高效RAG服务的系统优化框架。评估表明，相比基于LLM系统扩展构建的RAG系统，RAGO实现每芯片吞吐量提升2倍，首次响应延迟降低55%。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG), which combines large language models (LLMs) with retrievals from external knowledge databases, is emerging as a popular approach for reliable LLM serving. However, efficient RAG serving remains an open challenge due to the rapid emergence of many RAG variants and the substantial differences in workload characteristics across them. In this paper, we make three fundamental contributions to advancing RAG serving. First, we introduce RAGSchema, a structured abstraction that captures the wide range of RAG algorithms, serving as a foundation for performance optimization. Second, we analyze several representative RAG workloads with distinct RAGSchema, revealing significant performance variability across these workloads. Third, to address this variability and meet diverse performance requirements, we propose RAGO (Retrieval-Augmented Generation Optimizer), a system optimization framework for efficient RAG serving. Our evaluation shows that RAGO achieves up to a 2x increase in QPS per chip and a 55% reduction in time-to-first-token latency compared to RAG systems built on LLM-system extensions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。