用图模型增强检索生成,让大模型更懂知识间复杂关系。
GFM-RAG: Graph Foundation Model for Retrieval Augmented Generation
- 构建图神经网络,从知识图谱中推理复杂查询-知识关联。
- 在1400万三元组上训练,800万参数模型无需微调即适配新数据集。
- 多跳问答和领域专用任务均达顶尖性能,适合需要深度推理的场景。
检索增强生成(RAG)在将知识融入大语言模型方面表现优异,但传统RAG难以捕捉知识间的复杂关系,限制了其在多源知识整合推理中的表现。近期的GraphRAG通过构建图结构显式建模这些关系,提升了检索效率与效果,但仍受图结构噪声与不完整性的制约。为此,本文提出GFM-RAG,一种用于检索增强生成的图基础模型(GFM)。该模型采用创新的图神经网络,在大规模知识图谱(共60个,超过1400万三元组)和70万文档上进行两阶段训练,总参数量为800万。训练后的模型在未见数据集上无需微调即可实现优秀性能,展现出强大泛化能力。在三个多跳问答数据集及七个领域特定的RAG数据集上的实验表明,GFM-RAG达到当前最优性能,同时保持与神经尺度定律一致的效率,具备进一步优化潜力。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) has proven effective in integrating knowledge into large language models (LLMs). However, conventional RAGs struggle to capture complex relationships between pieces of knowledge, limiting their performance in intricate reasoning that requires integrating knowledge from multiple sources. Recently, graph-enhanced retrieval augmented generation (GraphRAG) builds graph structure to explicitly model these relationships, enabling more effective and efficient retrievers. Nevertheless, its performance is still hindered by the noise and incompleteness within the graph structure. To address this, we introduce GFM-RAG, a novel graph foundation model (GFM) for retrieval augmented generation. GFM-RAG is powered by an innovative graph neural network that reasons over graph structure to capture complex query-knowledge relationships. The GFM with 8M parameters undergoes a two-stage training process on large-scale datasets, comprising 60 knowledge graphs with over 14M triples and 700k documents. This results in impressive performance and generalizability for GFM-RAG, making it the first graph foundation model applicable to unseen datasets for retrieval without any fine-tuning required. Extensive experiments on three multi-hop QA datasets and seven domain-specific RAG datasets demonstrate that GFM-RAG achieves state-of-the-art performance while maintaining efficiency and alignment with neural scaling laws, highlighting its potential for further improvement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。