arXiv:2604.12487cs.CLcs.AI2026-04

用强化学习让大模型自主推理知识图谱,解决多跳问答难题

KG-Reasoner: A Reinforced Model for End-to-End Multi-Hop Knowledge Graph Reasoning

  • 将多步推理整合为统一思考过程,支持动态探索路径
  • 在8个基准上超越或媲美当前最佳方法,提升推理连贯性
  • 适合需要复杂知识推理的应用场景,如智能问答系统

大型语言模型在自然语言理解与生成方面表现强劲,但在知识密集型推理上仍存在不足。结构化知识图谱作为外部知识表示形式,在经典知识库问答任务中广泛应用。然而,针对复杂查询进行精确的多跳推理仍具挑战性。现有方法通常将推理过程分解为一系列孤立步骤,通过固定流程执行,限制了推理灵活性并导致中间信息丢失和决策不连贯。本文提出KG-Reasoner,一种端到端框架,将多步推理融入推理大模型的统一“思考”阶段。通过强化学习训练,使大模型内化知识图谱遍历过程,能够动态探索路径并在必要时回溯。在八个多跳及知识密集型推理基准上的实验表明,KG-Reasoner性能达到或超过当前最先进方法。代码已开源:https://github.com/Wangshuaiia/KG-Reasoner。

原文摘要 · Abstract (English)

Large Language Models (LLMs) exhibit strong abilities in natural language understanding and generation, yet they struggle with knowledge-intensive reasoning. Structured Knowledge Graphs (KGs) provide an effective form of external knowledge representation and have been widely used to enhance performance in classical Knowledge Base Question Answering (KBQA) tasks. However, performing precise multi-hop reasoning over KGs for complex queries remains highly challenging. Most existing approaches decompose the reasoning process into a sequence of isolated steps executed through a fixed pipeline. While effective to some extent, such designs constrain reasoning flexibility and fragment the overall decision process, often leading to incoherence and the loss of critical intermediate information from earlier steps. In this paper, we introduce KG-Reasoner, an end-to-end framework that integrates multi-step reasoning into a unified "thinking" phase of a Reasoning LLM. Through Reinforcement Learning (RL), the LLM is trained to internalize the KG traversal process, enabling it to dynamically explore reasoning paths, and perform backtracking when necessary. Experiments on eight multi-hop and knowledge-intensive reasoning benchmarks demonstrate that KG-Reasoner achieves competitive or superior performance compared to the state-of-the-art methods. Codes are available at the repository: https://github.com/Wangshuaiia/KG-Reasoner.

知识图谱多跳推理强化学习大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。