通过复用相似子图的计算结果，显著加速图结构RAG推理
SubGCache: Accelerating Graph-based RAG with Subgraph-level KV Cache
- 按子图结构聚类查询，为每类预计算并缓存键值对
- 在多个模型和框架上实现最高6.68倍的首次响应提速
- 适合需要快速响应的问答系统与知识密集型应用
基于图的检索增强生成（Graph-based RAG）使大语言模型通过图检索获取结构化知识作为上下文输入，提升推理准确性与上下文感知能力。我们发现不同查询可能检索到相似的子图作为提示，因此提出SubGCache，旨在通过复用具有相似结构提示（即子图）的查询间的计算来降低推理延迟。具体而言，SubGCache基于子图嵌入对查询进行聚类，为每类构建代表性子图，并预先计算其键值（KV）缓存。对于属于某类的查询，若其检索到的子图位于该类中，则直接复用该类代表性子图的预计算KV缓存，无需重新计算KV张量以节省计算开销。在两个新数据集上，针对多种LLM主干和图结构RAG框架的实验表明，SubGCache在保持甚至提升生成质量的同时，持续降低推理延迟，首次响应时间（TTFT）最高可减少6.68倍。
原文摘要 · Abstract (English)
Graph-based retrieval-augmented generation (RAG) enables large language models (LLMs) to incorporate structured knowledge via graph retrieval as contextual input, enhancing more accurate and context-aware reasoning. We observe that for different queries, it could retrieve similar subgraphs as prompts, and thus we propose SubGCache, which aims to reduce inference latency by reusing computation across queries with similar structural prompts (i.e., subgraphs). Specifically, SubGCache clusters queries based on subgraph embeddings, constructs a representative subgraph for each cluster, and pre-computes the key-value (KV) cache of the representative subgraph. For each query with its retrieved subgraph within a cluster, it reuses the pre-computed KV cache of the representative subgraph of the cluster without computing the KV tensors again for saving computation. Experiments on two new datasets across multiple LLM backbones and graph-based RAG frameworks demonstrate that SubGCache consistently reduces inference latency with comparable and even improved generation quality, achieving up to 6.68$\times$ reduction in time-to-first-token (TTFT).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。