让大模型先对齐知识再推理,提升多跳问答准确率
Multi-hop Reasoning via Early Knowledge Alignment
- 先用检索知识对齐模型,再规划推理路径
- 在6个数据集上提升检索精度,减少错误累积
- 无需训练,适配大模型,通用性强
检索增强生成(RAG)已成为大语言模型应对需要领域或最新知识的复杂查询的强大范式。为解决多跳问题,已有基于强化学习的迭代RAG方法被提出。然而,现有系统在规划时未考虑可检索语料库信息,导致检索和推理链效率低下,性能受限。本文提出早期知识对齐(EKA)模块,在迭代RAG中引入上下文相关的检索知识,使模型在规划前即与检索集对齐。在六个标准RAG数据集上的实验表明,EKA显著提升检索精度,减少错误传播,增强性能与效率。熵分析显示,早期知识对齐减少了推理过程中的无效探索,使模型更聚焦于相关知识子集。EKA作为无需训练的通用推理策略,可无缝扩展至大模型,跨数据集与检索语料的泛化测试验证其鲁棒性。整体上,EKA推动了迭代RAG的前沿进展,并揭示了结构化推理与高效探索在强化学习增强框架中的关键协同作用。代码已开源。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for Large Language Models (LLMs) to address knowledge-intensive queries requiring domain-specific or up-to-date information. To handle complex multi-hop questions that are challenging for single-step retrieval, iterative RAG approaches incorporating reinforcement learning have been proposed. However, existing iterative RAG systems typically plan to decompose questions without leveraging information about the available retrieval corpus, leading to inefficient retrieval and reasoning chains that cascade into suboptimal performance. In this paper, we introduce Early Knowledge Alignment (EKA), a simple but effective module that aligns LLMs with retrieval set before planning in iterative RAG systems with contextually relevant retrieved knowledge. Extensive experiments on six standard RAG datasets demonstrate that by establishing a stronger reasoning foundation, EKA significantly improves retrieval precision, reduces cascading errors, and enhances both performance and efficiency. Our analysis from an entropy perspective demonstrate that incorporating early knowledge reduces unnecessary exploration during the reasoning process, enabling the model to focus more effectively on relevant information subsets. Moreover, EKA proves effective as a versatile, training-free inference strategy that scales seamlessly to large models. Generalization tests across diverse datasets and retrieval corpora confirm the robustness of our approach. Overall, EKA advances the state-of-the-art in iterative RAG systems while illuminating the critical interplay between structured reasoning and efficient exploration in reinforcement learning-augmented frameworks. The code is released at \href{https://github.com/yxzwang/EarlyKnowledgeAlignment}{Github}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。