协调大模型生成与数据库检索，提升异构RAG服务效率
HedraRAG: Coordinating LLM Generation and Database Retrieval in Heterogeneous RAG Serving
- 用图结构抽象多阶段RAG流程，动态调整执行路径
- 实测速度提升1.5至5倍，显著降低延迟
- 适合高并发、复杂请求的生产级RAG系统部署
本文针对异构检索增强生成（RAG）服务中日益突出的系统级挑战展开研究，这些问题源于复杂的多阶段工作流和多样化的请求模式，导致执行效率低下。我们提出HedraRAG，一个基于图抽象的运行时系统，通过暴露阶段内并行性、请求内相似性及请求间偏斜度等优化机会，实现动态图变换——包括节点拆分、重排、边添加与依赖重连，作用于跨越并发请求的子图波前。生成的执行计划被映射到混合CPU-GPU流水线，从而提升资源利用率并减少延迟。在多种RAG工作流上的评估表明，相比现有框架，速度提升超过1.5倍，最高可达5倍，验证了生成与检索协同调度在服务环境中的有效性。
原文摘要 · Abstract (English)
This paper addresses emerging system-level challenges in heterogeneous retrieval-augmented generation (RAG) serving, where complex multi-stage workflows and diverse request patterns complicate efficient execution. We present HedraRAG, a runtime system built on a graph-based abstraction that exposes optimization opportunities across stage-level parallelism, intra-request similarity, and inter-request skewness. These opportunities are realized through dynamic graph transformations, such as node splitting, reordering, edge addition, and dependency rewiring, applied to wavefronts of subgraphs spanning concurrent requests. The resulting execution plans are mapped onto hybrid CPU-GPU pipelines to improve resource utilization and reduce latency. Evaluations across a wide range of RAG workflows demonstrate speedups exceeding 1.5x and reaching up to 5x over existing frameworks, showcasing the effectiveness of coordinated generation and retrieval in serving environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。