让大模型在知识图谱上辩论式推理,解决幻觉与错误路径问题。
Debate on Graph: a Flexible and Reliable Reasoning Framework for Large Language Models
- 引入多角色辩论机制,逐步拆解复杂问题
- 每步推理后尝试答案,有效缩短冗长推理链
- 兼容多种大模型,适用于知识问答场景
大型语言模型在实际应用中因缺乏相关知识易产生幻觉。相比之下,知识图谱包含大量结构化符号事实。将两者结合已成为研究热点,知识图谱问答(KGQA)是重要验证任务。现有方法面临两大挑战:推理路径过长干扰答案生成,以及假阳性关系阻碍路径优化。本文提出一种迭代交互式KGQA框架——图上辩论(DoG),利用大模型的交互学习能力进行推理与辩论。DoG采用子图聚焦机制,使模型在每步推理后尝试生成答案,从而缓解长路径影响;同时通过多角色辩论团队逐步简化复杂问题,降低假阳性关系的影响。实验在五个公开数据集上验证了有效性,显著优于当前最优方法ToG,WebQuestions和GrailQA准确率分别提升23.7%和9.1%。跨模型集成实验也证明了其灵活性。代码已开源。
原文摘要 · Abstract (English)
Large Language Models (LLMs) may suffer from hallucinations in real-world applications due to the lack of relevant knowledge. In contrast, knowledge graphs encompass extensive, multi-relational structures that store a vast array of symbolic facts. Consequently, integrating LLMs with knowledge graphs has been extensively explored, with Knowledge Graph Question Answering (KGQA) serving as a critical touchstone for the integration. This task requires LLMs to answer natural language questions by retrieving relevant triples from knowledge graphs. However, existing methods face two significant challenges: \textit{excessively long reasoning paths distracting from the answer generation}, and \textit{false-positive relations hindering the path refinement}. In this paper, we propose an iterative interactive KGQA framework that leverages the interactive learning capabilities of LLMs to perform reasoning and Debating over Graphs (DoG). Specifically, DoG employs a subgraph-focusing mechanism, allowing LLMs to perform answer trying after each reasoning step, thereby mitigating the impact of lengthy reasoning paths. On the other hand, DoG utilizes a multi-role debate team to gradually simplify complex questions, reducing the influence of false-positive relations. This debate mechanism ensures the reliability of the reasoning process. Experimental results on five public datasets demonstrate the effectiveness and superiority of our architecture. Notably, DoG outperforms the state-of-the-art method ToG by 23.7\% and 9.1\% in accuracy on WebQuestions and GrailQA, respectively. Furthermore, the integration experiments with various LLMs on the mentioned datasets highlight the flexibility of DoG. Code is available at \url{https://github.com/reml-group/DoG}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。