arXiv:2601.01195cs.AI2026-01被引 2

用强化学习优化时间知识图谱问答的多跳推理路径。

Reinforcement Learning Enhanced Multi-hop Reasoning for Temporal Knowledge Question Answering

  • 通过提示工程生成多样推理路径,冷启动微调
  • 引入树状结构策略优化,提升复杂查询准确率
  • 增强可解释性,适合处理带时间约束的问答任务

时间知识图谱问答(TKGQA)需在知识图谱中对具有时间约束的实体关系进行多跳推理以回答问题。然而,在每一步推理中,大语言模型(LLMs)会检索包含大量时间相似且语义复杂的关联,导致次优决策和错误传播。为此,我们提出多跳推理增强(MRE)框架,同时优化前向与后向推理,以识别全局最优推理轨迹。具体而言,MRE首先通过提示工程引导LLM生成多样推理路径;有效路径经筛选后用于监督微调,作为冷启动策略;最后引入树状分组相对策略优化(T-GRPO),一种递归的、树状结构的探索式学习方法。每一步的探索均与前一步建立强因果依赖,评估则基于后续步骤的多路径探索反馈。在两个TKGQA基准上的实验表明,所提MRE模型在处理复杂多跳查询时持续优于现有最先进方法。进一步分析显示其具备更强可解释性和对噪声时间标注的鲁棒性。

原文摘要 · Abstract (English)

Temporal knowledge graph question answering (TKGQA) involves multi-hop reasoning over temporally constrained entity relationships in the knowledge graph to answer a given question. However, at each hop, large language models (LLMs) retrieve subgraphs with numerous temporally similar and semantically complex relations, increasing the risk of suboptimal decisions and error propagation. To address these challenges, we propose the multi-hop reasoning enhanced (MRE) framework, which enhances both forward and backward reasoning to improve the identification of globally optimal reasoning trajectories. Specifically, MRE begins with prompt engineering to guide the LLM in generating diverse reasoning trajectories for a given question. Valid reasoning trajectories are then selected for supervised fine-tuning, serving as a cold-start strategy. Finally, we introduce Tree-Group Relative Policy Optimization (T-GRPO), a recursive, tree-structured learning-by-exploration approach. At each hop, exploration establishes strong causal dependencies on the previous hop, while evaluation is informed by multi-path exploration feedback from subsequent hops. Experimental results on two TKGQA benchmarks indicate that the proposed MRE-based model consistently surpasses state-of-the-art (SOTA) approaches in handling complex multi-hop queries. Further analysis highlights improved interpretability and robustness to noisy temporal annotations.

知识图谱多跳推理强化学习时间问答

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。