arXiv:2501.08686cs.DBcs.CL2025-01被引 24

用知识图谱增强大模型,解决复杂数据映射中的语义歧义和幻觉问题。

Knowledge Graph-based Retrieval-Augmented Generation for Schema Matching

  • 通过向量、图遍历和查询三种方式从外部知识图谱中检索相关子图
  • 在MIMIC和Synthea数据集上精度和F1提升超30%,且无需重新训练
  • 适合需要高准确率的医疗等专业领域数据集成场景

传统基于相似度的模式匹配方法因缺乏常识和领域知识,在特定复杂映射场景中难以解决语义歧义与冲突。大语言模型(LLMs)的幻觉问题也使其在模式匹配中面临挑战。为此,我们提出基于知识图谱的检索增强生成模型KG-RAG4SM。该模型引入基于向量、图遍历和查询的三种图谱检索机制,以及混合检索与排序策略,从外部大型知识图谱中识别最相关的子图。实验表明,该方法可在不重新训练的前提下,显著提升复杂匹配任务的准确性。在MIMIC数据集上,相较于基于LLM的SOTA方法(如Jellyfish-8B),KG-RAG4SM的精度和F1分别提升35.89%和30.50%;在Synthea数据集上,使用GPT-4o-mini时,相比基于PLM的SOTA方法(如SMAT),精度和F1分别提升69.20%和21.97%。结果还显示,该方法在端到端模式匹配中更高效,可扩展至大规模知识图谱。真实场景案例研究证实,该方案有效缓解了大模型在模式匹配中的幻觉问题。

原文摘要 · Abstract (English)

Traditional similarity-based schema matching methods are incapable of resolving semantic ambiguities and conflicts in domain-specific complex mapping scenarios due to missing commonsense and domain-specific knowledge. The hallucination problem of large language models (LLMs) also makes it challenging for LLM-based schema matching to address the above issues. Therefore, we propose a Knowledge Graph-based Retrieval-Augmented Generation model for Schema Matching, referred to as the KG-RAG4SM. In particular, KG-RAG4SM introduces novel vector-based, graph traversal-based, and query-based graph retrievals, as well as a hybrid approach and ranking schemes that identify the most relevant subgraphs from external large knowledge graphs (KGs). We showcase that KG-based retrieval-augmented LLMs are capable of generating more accurate results for complex matching cases without any re-training. Our experimental results show that KG-RAG4SM outperforms the LLM-based state-of-the-art (SOTA) methods (e.g., Jellyfish-8B) by 35.89% and 30.50% in terms of precision and F1 score on the MIMIC dataset, respectively; KG-RAG4SM with GPT-4o-mini outperforms the pre-trained language model (PLM)-based SOTA methods (e.g., SMAT) by 69.20% and 21.97% in terms of precision and F1 score on the Synthea dataset, respectively. The results also demonstrate that our approach is more efficient in end-to-end schema matching, and scales to retrieve from large KGs. Our case studies on the dataset from the real-world schema matching scenario exhibit that the hallucination problem of LLMs for schema matching is well mitigated by our solution.

知识图谱大模型数据集成检索增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。