提出可感知知识图谱结构的累积奖励模型,提升复杂推理的准确性与安全性。
SCPRM: A Schema-aware Cumulative Process Reward Model for Knowledge Graph Question Answering

- 基于推理前缀和模式距离设计累积奖励机制
- 在医疗与法律知识图谱问答中平均提升Hits@k 1.18%
- 适合对错误容忍度低的高风险推理任务
大型语言模型在复杂推理中表现优异,但评估其中间步骤仍具挑战。尽管过程奖励模型提供逐步监督,却常出现风险补偿效应——错误步骤被后续正确步骤抵消,导致错误推理路径获得高分。这一问题在知识图谱(KG)推理中尤为突出,因起点与终点间可能存在多条路径,一步失误即可能导致整个路径失效。此类缺陷在医疗、法律等高风险任务中尤为危险。为此,我们提出一种感知模式的累积过程奖励模型(SCPRM),通过结合推理前缀与从查询解析出的隐含目标之间的模式距离,实现对推理路径的累积性未来奖励评估,以引导路径探索。进一步将SCPRM集成至蒙特卡洛树搜索(MCTS)中,形成SCPRM-MCTS,用于知识图谱问答(KGQA)的多跳推理。在医疗与法律领域知识图谱问答及CWQ数据集上,相较于强基线,SCPRM-MCTS平均提升Hits@k 1.18%,验证了其更准确、更具风险敏感性的推理评估能力。
原文摘要 · Abstract (English)
Large language models excel at complex reasoning, yet evaluating their intermediate steps remains challenging. Although process reward models provide step-wise supervision, they often suffer from a risk compensation effect, where incorrect steps are offset by later correct ones, assigning high rewards to flawed reasoning paths. This issue is further exacerbated in knowledge graph (KG) reasoning, as there may exist multiple paths between the start and end entities in the KGs, and a risky step can make the reasoning path flawed. Those limitations are problematic in risk-sensitive tasks such as medical and legal KG reasoning. To address the issues, we propose a Schema-aware Cumulative Process Reward Model (SCPRM) that evaluates reasoning paths by conditioning on the reasoning prefix , and incorporating schema distance between current reasoning step and the implicit target parsed from the query, which provides cumulative and future rewards to guide the path explorations. We further integrate SCPRM into Monte Carlo Tree Search (MCTS) as SCPRM-MCTS to conduct multi-hop reasoning on KGs for question answering (QA) tasks. Across medical and legal KGQA and CWQ, SCPRM-MCTS improves the performance of Hits@k by an average of 1.18% over strong baselines, demonstrating more accurate and risk-sensitive reasoning evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。