用统计方法建图并生成推理路径,提升大模型问答准确性
AGRAG: Advanced Graph-based Retrieval-Augmented Generation for LLMs
- 用统计替代大模型实体抽取,避免幻觉导致的图构建错误
- 设计最小成本最大影响力子图算法,生成更全面的推理路径
- 支持循环结构,适合复杂逻辑推理,适合需要精准解释的任务
基于图的检索增强生成(Graph-based RAG)在为大语言模型(LLMs)注入结构化知识方面展现出巨大潜力。然而现有方法面临三大挑战:由大模型幻觉引起的图构建不准确;因缺乏明确推理理由导致的推理能力弱;以及由于推理不足造成回答不完整,使其在某些任务上表现不及NaiveRAG。为此,我们提出AGRAG,一个先进的图基检索增强生成框架。在图构建阶段,AGRAG以基于统计的方法替代广泛使用的大模型实体提取,避免幻觉与错误传播。在检索阶段,将图推理过程建模为最小成本最大影响力(MCMI)子图生成问题,旨在包含高影响节点的同时最小化边成本,使生成的推理路径更全面。我们证明该问题是NP-hard,并提出一种贪心算法求解。生成的MCMI子图可作为显式推理路径,指导大模型关注相关文本内容,降低噪声干扰,提升推理能力。相比简单的树状路径,MCMI子图允许更复杂的图结构(如环),增强推理路径的完整性。代码与提示已开源:https://github.com/Wyb0627/AGRAG。
原文摘要 · Abstract (English)
Graph-based retrieval-augmented generation (Graph-based RAG) has demonstrated significant potential in enhancing Large Language Models (LLMs) with structured knowledge. However, existing methods face three critical challenges: Inaccurate Graph Construction, caused by LLM hallucination; Poor Reasoning Ability, caused by failing to generate explicit reasons telling LLM why certain chunks were selected; and Inadequate Answering, which only partially answers the query due to the inadequate LLM reasoning, making their performance lag behind NaiveRAG on certain tasks. To address these issues, we propose AGRAG, an advanced graph-based retrieval-augmented generation framework. When constructing the graph, AGRAG substitutes the widely used LLM entity extraction method with a statistics-based method, avoiding hallucination and error propagation. During retrieval, AGRAG formulates the graph reasoning procedure as the Minimum Cost Maximum Influence (MCMI) subgraph generation problem, where we try to include more nodes with high influence score, but with less involving edge cost, to make the generated reasoning paths more comprehensive. We prove this problem to be NP-hard, and propose a greedy algorithm to solve it. The MCMI subgraph generated can serve as explicit reasoning paths to tell LLM why certain chunks were retrieved, thereby making the LLM better focus on the query-related part contents of the chunks, reducing the impact of noise, and improving AGRAG's reasoning ability. Furthermore, compared with the simple tree-structured reasoning paths, our MCMI subgraph can allow more complex graph structures, such as cycles, and improve the comprehensiveness of the generated reasoning paths. The code and prompt of AGRAG are released at: https://github.com/Wyb0627/AGRAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。