用结构化图增强大模型，减少幻觉
Beyond Textual Context: Structural Graph Encoding with Adaptive Space Alignment to alleviate the hallucination of LLMs
- 设计新架构，同时利用知识图谱的结构与语义信息
- 在多个数据集上显著提升事实推理准确率
- 适合需要高可靠性生成的场景，如医疗问答
当前大语言模型缓解幻觉的主要方法是引入知识图谱（KGs），但通常将KG视为纯文本，仅提取语义信息，忽略其关键结构特征。此外，KG编码器与大模型文本嵌入之间的空间差异也阻碍了结构化知识的有效融合。为此，我们提出SSKG-LLM，一种创新模型架构，可将知识图谱的结构与语义信息高效融入大模型推理过程。该架构包含知识图谱检索（KGR）模块和知识图谱编码（KGE）模块，以保留语义并利用结构；同时引入知识图谱适配（KGA）模块，使大模型能理解图谱嵌入。我们在多个基准上进行广泛实验并深入分析，结果表明，引入图谱结构信息可显著提升大模型的事实推理能力。代码已开源。
原文摘要 · Abstract (English)
Currently, the main approach for Large Language Models (LLMs) to tackle the hallucination issue is incorporating Knowledge Graphs(KGs).However, LLMs typically treat KGs as plain text, extracting only semantic information and limiting their use of the crucial structural aspects of KGs. Another challenge is the gap between the embedding spaces of KGs encoders and LLMs text embeddings, which hinders the effective integration of structured knowledge. To overcome these obstacles, we put forward the SSKG-LLM, an innovative model architecture that is designed to efficiently integrate both the Structural and Semantic information of KGs into the reasoning processes of LLMs. SSKG-LLM incorporates the Knowledge Graph Retrieval (KGR) module and the Knowledge Graph Encoding (KGE) module to preserve semantics while utilizing structure. Then, the Knowledge Graph Adaptation (KGA) module is incorporated to enable LLMs to understand KGs embeddings. We conduct extensive experiments and provide a detailed analysis to explore how incorporating the structural information of KGs can enhance the factual reasoning abilities of LLMs. Our code are available at https://github.com/yfangZhang/SSKG-LLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。