用知识图谱引导大模型多跳推理，让回答更符合逻辑路径。
KG-o1: Enhancing Multi-hop Question Answering in Large Language Models via Knowledge Graph Integration
- 引入知识图谱构建逻辑路径，约束模型推理方向。
- 在复杂数据集上提升准确率，优于现有长链推理模型。
- 适合需要严谨逻辑推理的研究者与应用开发人员。
大语言模型（LLM）在多跳问答等知识密集型任务中表现不佳，因其生成的思维链（CoTs）常偏离真实推理路径。知识图谱（KG）则显式表达实体间逻辑关系，弥补这一差距。同时，长链推理模型（如o1）证明了长步骤推理可显著提升性能。为此，我们提出KG-o1，一种四阶段方法：首先筛选初始实体并生成复杂子图；其次构建子图逻辑路径，创建包含复杂发散思考过程的数据集，训练模型模仿长期推理；最后通过拒绝采样生成自优化语料，用于直接偏好优化（DPO），进一步强化推理能力。在两个简单和两个复杂数据集上的实验表明，KG-o1模型在所有任务中均优于现有长链推理模型。
原文摘要 · Abstract (English)
Large Language Models (LLMs) face challenges in knowledge-intensive reasoning tasks like classic multi-hop question and answering, which involves reasoning across multiple facts. This difficulty arises because the chain of thoughts (CoTs) generated by LLMs in such tasks often deviate from real or a priori reasoning paths. In contrast, knowledge graphs (KGs) explicitly represent the logical connections between facts through entities and relationships. This reflects a significant gap. Meanwhile, large reasoning models (LRMs), such as o1, have demonstrated that long-step reasoning significantly enhances the performance of LLMs. Building on these insights, we propose KG-o1, a four-stage approach that integrates KGs to enhance the multi-hop reasoning abilities of LLMs. We first filter out initial entities and generate complex subgraphs. Secondly, we construct logical paths for subgraphs and then use knowledge graphs to build a dataset with a complex and extended brainstorming process, which trains LLMs to imitate long-term reasoning. Finally, we employ rejection sampling to generate a self-improving corpus for direct preference optimization (DPO), further refining the LLMs reasoning abilities. We conducted experiments on two simple and two complex datasets. The results show that KG-o1 models exhibit superior performance across all tasks compared to existing LRMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。