让大模型更准确理解知识图谱结构,避免推理时‘乱猜’。
Structure-Internalized Rule Language Model for Faithful Knowledge Graph Reasoning

- 用结构规则生成器把图谱逻辑嵌入大模型,让其理解图结构约束。
- 在36个数据集上优于17种主流方法,显著提升推理准确性与可信度。
- 适合需要高可靠性的知识推理场景,如医疗、金融决策支持。
知识图谱推理(KGR)旨在利用图谱中的结构证据发现隐含事实,这对模型的结构语义理解能力提出挑战。近期研究显示,大语言模型(LLMs)通过灵活的上下文学习在KGR任务中取得显著进展,但其参数化知识与图谱结构上下文之间存在本质表示不一致,导致模型难以感知符合图谱约束的推理证据,削弱了推理的有效性与可信度,这一问题称为‘推理证据感知漂移’。为此,我们提出结构内嵌规则语言模型(SIRLM),以结构规则生成为核心,将结构知识的参数化学习与推理逻辑的可信度评估相耦合,使模型能紧密锚定于图谱依据。具体地,设计结构内嵌规则生成器(SIRG),融合增强型上下文学习模块与结构关系记忆,协调结构与参数知识;引入基于结构不变性学习的图谱分词器及基于规则约束的消息传播神经符号推理器,分别提供可学习的结构表征与忠实的规则执行反馈。SIRLM可无缝集成至标准大模型训练范式(如SFT和GRPO)。在36个数据集上对比17种先进KGR方法的实验表明,SIRLM表现显著优越。
原文摘要 · Abstract (English)
Knowledge Graph Reasoning (KGR) aims to discover latent facts by leveraging the structural evidence available in KGs, posing a challenge to the structural semantic understanding capability of KGR models. Recent studies have demonstrated that Large Language Models (LLMs) can achieve remarkable progress on KGR tasks via flexible in-context learning. However, the inherent representation inconsistency between KG structural context and LLM parametric knowledge remains inadequately addressed. This limitation prevents LLMs from effectively perceiving reasoning evidence that aligns with KG constraints, which undermines both the effectiveness and faithfulness of reasoning. We refer to this problem as reasoning evidence perception drift of LLMs over KGs. To address this problem, we propose a Structure-Internalized Rule Language Model (SIRLM), which centers on structural rule generation to couple the parametric learning of structural knowledge with the faithfulness evaluation of reasoning logic, enabling LLMs to anchor tightly to KG-grounded evidence. Specifically, we first design a Structure-Internalized Rule Generator (SIRG), which incorporates an in-context learning block augmented with a structural relation memory to coordinate structural and parametric knowledge. Furthermore, we equip SIRG with a KG tokenizer based on structural invariance learning and a neuro-symbolic reasoner based on rule-constrained message propagation. These components provide SIRG with learnable structural representations and faithful rule-execution feedback, respectively. Our SIRLM can be seamlessly integrated into standard LLM training paradigms, such as SFT and GRPO. Extensive experiments against 17 state-of-the-art KGR methods on 36 datasets demonstrate the significant superiority of SIRLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。