开源真实RAG服务数据集,助力优化大模型问答系统性能
RAGPulse: An Open-Source RAG Workload Trace to Optimize RAG Serving Systems
- 基于高校问答系统采集真实请求日志,还原RAG多阶段工作负载
- 发现请求存在显著时间局部性和热点文档访问集中(超80%请求集中在少数文档)
- 适合研究RAG缓存、批处理等优化策略的开发者与学术界使用
检索增强生成(RAG)是构建可靠、知识密集型大语言模型应用的关键范式。然而,RAG系统复杂的多阶段流水线(检索、生成)和独特的负载特征(如知识依赖性)给服务性能优化带来巨大挑战。现有通用LLM推理追踪无法捕捉RAG特有的动态,导致学术研究与实际部署间存在显著性能差距。为此,本文提出RAGPulse,一个开源的RAG工作负载追踪数据集。该数据集来自自2024年4月起服务于超过4万名师生的高校级问答系统。我们详细描述了RAGPulse的系统架构、隐私保护的哈希数据格式,并提供了深入的统计分析。分析显示,真实RAG负载表现出显著的时间局部性,且文档访问高度倾斜——超过80%的请求集中于少数热点文档。RAGPulse为研究人员开发和验证新型RAG优化策略(如内容感知批处理、检索缓存)提供了高保真基础,最终提升RAG服务的效率与可靠性。代码已开源:https://github.com/flashserve/RAGPulse。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) is a critical paradigm for building reliable, knowledge-intensive Large Language Model (LLM) applications. However, the multi-stage pipeline (retrieve, generate) and unique workload characteristics (e.g., knowledge dependency) of RAG systems pose significant challenges for serving performance optimization. Existing generic LLM inference traces fail to capture these RAG-specific dynamics, creating a significant performance gap between academic research and real-world deployment. To bridge this gap, this paper introduces RAGPulse, an open-source RAG workload trace dataset. This dataset was collected from an university-wide Q&A system serving that has served more than 40,000 students and faculties since April 2024. We detail RAGPulse's system architecture, its privacy-preserving hash-based data format, and provide an in-depth statistical analysis. Our analysis reveals that real-world RAG workloads exhibit significant temporal locality and a highly skewed hot document access pattern. RAGPulse provides a high-fidelity foundation for researchers to develop and validate novel optimization strategies for RAG systems, such as content-aware batching and retrieval caching, ultimately enhancing the efficiency and reliability of RAG services. The code is available at https://github.com/flashserve/RAGPulse.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。