用知识图谱路径约束大模型推理，提升事实准确性
Follow the Path: Reasoning over Knowledge Graph Paths to Improve Large Language Model Factuality
- 从大模型收集推理路径，用知识图谱进行事实校准
- 在6个复杂问答基准上提升6-14个百分点（pass@16）
- 对复杂问题和数值答案效果更显著，小模型提升最明显
我们提出fs1，一种简单而有效的方法，通过收集大模型的推理路径并将其锚定在知识图谱（KG）路径上，提升推理过程的事实性。在3.9K条具备事实依据的推理路径上微调8个指令微调的大语言模型，并在涵盖23.9K个问题的6个复杂开放域问答基准上严格评估。结果表明，fs1微调模型在所有任务上均优于使用并行采样的基线模型，绝对性能提升6-14个百分点（pass@16）。详细分析显示，fs1在需要3跳以上知识图谱路径的复杂问题以及数值类答案上表现尤为突出。此外，在单次推理中，较小的LLM获得最大提升。以往工作主要验证推理路径在STEM领域的有效性，本研究则表明，将推理锚定于真实知识图谱路径是实现可靠知识密集型任务的关键步骤。
原文摘要 · Abstract (English)
We introduce fs1, a simple yet effective method that improves the factuality of reasoning traces by collecting them from large reasoning models and grounding them in knowledge graph (KG) paths. We fine-tune eight instruction-tuned Large Language Models (LLMs) on 3.9K factually grounded reasoning traces and rigorously evaluate them on six complex open-domain question-answering (QA) benchmarks encompassing 23.9K questions. Our results demonstrate that our fs1-tuned model consistently outperforms instruction-tuned counterparts with parallel sampling by 6-14 absolute points (pass@16). Our detailed analysis shows that fs1 considerably improves model performance over more complex questions (requiring 3 or more hops on KG paths) and numerical answer types compared to the baselines. Furthermore, in single-pass inference, we notice that smaller LLMs show the most improvements. While prior works demonstrate the effectiveness of reasoning traces primarily in the STEM domains, our work shows strong evidence that anchoring reasoning to factual KG paths is a critical step in transforming LLMs for reliable knowledge-intensive tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。