让检索增强生成系统高效应对不断增长的文档库。
EraRAG: Efficient and Incremental Retrieval Augmented Generation for Growing Corpora
- 用分层图结构+基于超平面的局部敏感哈希,实现新数据快速插入
- 更新时间与令牌消耗降低一个数量级,准确率更高
- 适合需要持续更新知识库的实时应用,如智能客服、动态问答
基于图的检索增强生成(Graph-RAG)通过外部语料库结构化检索提升大语言模型性能。然而,现有方法通常假设语料库静态,新文档到来时需昂贵的全图重建,限制其在动态环境中的可扩展性。为此,我们提出EraRAG,一种支持高效可扩展动态更新的多层Graph-RAG框架。该方法利用基于超平面的局部敏感哈希(LSH)将原始语料库分层组织为图结构,实现无需破坏原有拓扑的新数据高效局部插入。设计避免了重训练或高成本重计算,同时保持高检索精度与低延迟。大规模基准测试表明,EraRAG相比现有Graph-RAG系统,在更新时间与令牌消耗上最多降低一个数量级,且准确率更优。本工作为需持续处理增长语料库的RAG系统提供了可行路径,弥合了检索效率与适应性之间的差距。代码与数据见https://github.com/EverM0re/EraRAG-Official。
原文摘要 · Abstract (English)
Graph-based Retrieval-Augmented Generation (Graph-RAG) enhances large language models (LLMs) by structuring retrieval over an external corpus. However, existing approaches typically assume a static corpus, requiring expensive full-graph reconstruction whenever new documents arrive, limiting their scalability in dynamic, evolving environments. To address these limitations, we introduce EraRAG, a novel multi-layered Graph-RAG framework that supports efficient and scalable dynamic updates. Our method leverages hyperplane-based Locality-Sensitive Hashing (LSH) to partition and organize the original corpus into hierarchical graph structures, enabling efficient and localized insertions of new data without disrupting the existing topology. The design eliminates the need for retraining or costly recomputation while preserving high retrieval accuracy and low latency. Experiments on large-scale benchmarks demonstrate that EraRag achieves up to an order of magnitude reduction in update time and token consumption compared to existing Graph-RAG systems, while providing superior accuracy performance. This work offers a practical path forward for RAG systems that must operate over continually growing corpora, bridging the gap between retrieval efficiency and adaptability. Our code and data are available at https://github.com/EverM0re/EraRAG-Official.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。