实证研究何时何地用知识图谱增强生成效果最佳
A Pilot Empirical Study on When and How to Use Knowledge Graphs as Retrieval Augmented Generation
- 对比9种配置在9个数据集上测试6种KG-RAG方法
- 发现不同场景下性能差异显著，配置影响大
- 适合想系统部署KG-RAG的研究者参考
将知识图谱（KG）融入检索增强生成（RAG）框架已引起广泛关注，早期研究显示其能缓解幻觉并提升准确率。然而，对快速涌现的KG-RAG方法尚缺乏系统性理解与比较分析。本文旨在为回答‘何时及如何使用KG-RAG’奠定基础，通过分析不同技术配置下的应用效果展开初步实证研究。我们梳理了KG-RAG框架的思维导图并总结主流流程，重新实现并评估了6种KG-RAG方法，在涵盖多个领域和场景的9个数据集上进行测试，分析了9种KG-RAG配置与17种大模型组合的影响，并首次尝试将元认知引入KG-RAG。结果表明，恰当的应用条件与组件最优配置至关重要。
原文摘要 · Abstract (English)
The integration of Knowledge Graphs (KGs) into the Retrieval Augmented Generation (RAG) framework has attracted significant interest, with early studies showing promise in mitigating hallucinations and improving model accuracy. However, a systematic understanding and comparative analysis of the rapidly emerging KG-RAG methods are still lacking. This paper seeks to lay the foundation for systematically answering the question of when and how to use KG-RAG by analyzing their performance in various application scenarios associated with different technical configurations. After outlining the mind map using KG-RAG framework and summarizing its popular pipeline, we conduct a pilot empirical study of KG-RAG works to reimplement and evaluate 6 KG-RAG methods across 9 datasets in diverse domains and scenarios, analyzing the impact of 9 KG-RAG configurations in combination with 17 LLMs, and combining Metacognition with KG-RAG as a pilot attempt. Our results underscore the critical role of appropriate application conditions and optimal configurations of KG-RAG components.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。