让大模型与知识图谱协同推理,解决开放域知识发现中的幻觉问题。
Knowledge Reasoning Language Model: Unifying Knowledge and Language for Inductive Knowledge Graph Reasoning
- 设计新型指令格式与分词器,对齐大模型与知识图谱表示
- 通过动态记忆机制融合模型内生知识与外部图谱上下文
- 结构感知预测器严格约束推理结果,提升可信度
归纳式知识图谱推理(Inductive KGR)旨在发现包含未知实体与关系的开放域知识图谱中的事实,这对模型理解不确定的图谱成分提出了挑战。现有研究提出知识图谱基础模型(KGFMs)以学习跨图谱的结构不变性来应对不确定性。近期大型语言模型(LLMs)在开放域知识推理中展现出强大能力,因此最新研究聚焦于将LLM知识与图谱上下文结合的基于LLM的KGFMs。然而,大模型的内在知识可能被稀疏的图谱上下文掩盖,导致知识扭曲,对推理造成不可逆损害。此外,现有方法仍难以充分控制生成幻觉,严重限制了推理结果的可信度。为此,我们提出知识推理语言模型(KRLM),实现整个推理过程中大模型知识与图谱上下文的统一协调。具体而言,设计知识推理语言(KRL)指令格式与分词器,对齐大模型知识与图谱表征;提出一种KRL注意力层,通过动态知识记忆机制协调内生知识与外部图谱上下文;最后设计结构感知的下一条目预测器,严格约束推理结果在可信知识域内。在25个真实世界归纳式KGR数据集上的实验表明,所提KRLM在零样本推理与微调场景下均显著优于现有方法。
原文摘要 · Abstract (English)
Inductive Knowledge Graph Reasoning (KGR) aims to discover facts in open-domain KGs containing unknown entities and relations, which poses a challenge for KGR models in comprehending uncertain KG components. Existing studies have proposed Knowledge Graph Foundation Models (KGFMs) that learn structural invariances across KGs to handle this uncertainty. Recently, Large Language Models (LLMs) have demonstrated strong capabilities for open-domain knowledge reasoning. As a result, the latest research has focused on LLM-based KGFMs that integrate LLM knowledge with KG context for inductive KGR. However, the intrinsic knowledge of LLMs may be overshadowed by sparse KG context, leading to LLM knowledge distortion, which can cause irreversible damage to model reasoning. Moreover, existing LLM-based KGR methods still struggle to fully constrain generative hallucinations in LLMs, severely limiting the credibility of reasoning results. To address these limitations, we propose a Knowledge Reasoning Language Model (KRLM) that achieves unified coordination between LLM knowledge and KG context throughout the KGR process. Specifically, we design a Knowledge Reasoning Language (KRL) instruction format and a KRL tokenizer to align LLM knowledge with KG representations. Then, we propose a KRL attention layer that coordinates intrinsic LLM knowledge with additional KG context through a dynamic knowledge memory mechanism. Finally, a structure-aware next-entity predictor is proposed, which strictly constrains the reasoning results within a trustworthy knowledge domain. Extensive experimental results on 25 real-world inductive KGR datasets demonstrate the significant superiority of the proposed KRLM\footnote{Our source codes are available at https://anonymous.4open.science/r/KRLM-EA36 in both zero-shot reasoning and fine-tuning scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。