让大模型分步推理更准，通过多视角知识图谱增强
Think Parallax: Solving Multi-Hop Problems via Multi-View Knowledge-Graph-Based Retrieval-Augmented Generation
- 将多跳推理拆解为多个语义视角，避免信息混淆
- 在WebQSP和CWQ上表现超越现有方法，幻觉率显著降低
- 适合需要精准多步推理的问答与科研场景
大型语言模型在基于知识图谱的多跳推理中仍面临挑战，我们发现一个被忽视的结构原因：Transformer注意力头在推理不同阶段自然专注于特定语义关系，形成逐跳对齐的传递模式。这一发现表明多跳推理本质上是多视角的，但现有基于知识图谱的检索增强生成（KG-RAG）系统将所有推理步骤压缩为单一表征空间，抑制了这种隐含结构，导致路径探索出现噪声或偏离。我们提出ParallaxRAG，一种对称的多视角框架，将查询与知识图谱解耦至对齐的头级语义空间。通过强制多个注意力头间的语义多样性，同时约束弱相关路径，ParallaxRAG构建出更准确、更干净的子图，引导模型进行有依据的逐跳推理。在WebQSP和CWQ数据集上达到当前最优的检索与问答性能，显著减少幻觉，并在生物医学领域的BioASQ基准上表现出强泛化能力。
原文摘要 · Abstract (English)
Large language models (LLMs) still struggle with multi-hop reasoning over knowledge-graphs (KGs), and we identify a previously overlooked structural reason for this difficulty: Transformer attention heads naturally specialize in distinct semantic relations across reasoning stages, forming a hop-aligned relay pattern. This key finding suggests that multi-hop reasoning is inherently multi-view, yet existing KG-based retrieval-augmented generation (KG-RAG) systems collapse all reasoning hops into a single representation, flat embedding space, suppressing this implicit structure and causing noisy or drifted path exploration. We introduce ParallaxRAG, a symmetric multi-view framework that decouples queries and KGs into aligned, head-specific semantic spaces. By enforcing relational diversity across multiple heads while constraining weakly related paths, ParallaxRAG constructs more accurate, cleaner subgraphs and guides LLMs through grounded, hop-wise reasoning. On WebQSP and CWQ, it achieves state-of-the-art retrieval and QA performance, substantially reduces hallucination, and generalizes strongly to the biomedical BioASQ benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。