测试知识图谱不完整时，大模型检索增强生成的效果如何。
Evaluating Knowledge Graph Based Retrieval Augmented Generation Methods under Knowledge Incompleteness
- 用不同方法删减知识图谱三元组，模拟真实不完整情况。
- 发现现有KG-RAG方法对知识缺失非常敏感，性能显著下降。
- 提醒研究者需开发更鲁棒的模型，适合实际应用环境。
基于知识图谱的检索增强生成（KG-RAG）通过从知识图谱（KG）中检索信息来提升大语言模型（LLM）在问答（QA）等任务中的表现。然而，现实中的知识图谱常存在信息缺失。现有评估基准未能充分反映知识图谱不完整性对KG-RAG性能的影响。本文通过采用不同策略移除知识图谱中的三元组，系统评估了在知识不完整条件下的KG-RAG方法表现。结果表明，当前KG-RAG方法对知识缺失高度敏感，凸显了在真实场景下发展更鲁棒方法的必要性。
原文摘要 · Abstract (English)
Knowledge Graph based Retrieval-Augmented Generation (KG-RAG) is a technique that enhances Large Language Model (LLM) inference in tasks like Question Answering (QA) by retrieving relevant information from knowledge graphs (KGs). However, real-world KGs are often incomplete, meaning that essential information for answering questions may be missing. Existing benchmarks do not adequately capture the impact of KG incompleteness on KG-RAG performance. In this paper, we systematically evaluate KG-RAG methods under incomplete KGs by removing triples using different methods and analyzing the resulting effects. We demonstrate that KG-RAG methods are sensitive to KG incompleteness, highlighting the need for more robust approaches in realistic settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。