arXiv:2512.09487cs.CLcs.AI2025-12ACL被引 4

用强化学习让大模型智能选择图文混合检索时机与内容。

RouteRAG: Efficient Retrieval-Augmented Generation from Text and Graph via Reinforcement Learning

  • 通过强化学习让模型自适应决定何时查文本、何时查图、何时输出答案。
  • 在5个问答数据集上超越现有基线,提升复杂推理准确率。
  • 适合需要高效多跳推理的智能问答系统研发者参考。

检索增强生成(RAG)将非参数化知识融入大语言模型,通常来自非结构化文本和结构化图谱。尽管近期进展已通过强化学习(RL)实现文本-based RAG 的多轮推理,但将该方法拓展至混合检索仍面临挑战。现有图谱或混合系统多依赖固定或人工设计的检索流程,难以在推理过程中动态补充证据。此外,虽然图谱提供对多跳推理至关重要的关系结构,但其检索成本显著更高。为此,我们提出 model{},一种基于强化学习的框架,支持大模型进行多轮且自适应的图文混合检索增强生成。 model{} 通过强化学习联合优化整个生成过程,使模型学会判断何时推理、从文本或图中检索什么内容、何时生成最终答案,所有决策均在统一生成策略中完成。为引导学习,我们设计了两阶段训练框架,兼顾任务结果与检索效率,使模型能有效利用混合证据同时避免不必要的检索开销。在五个问答基准上的实验表明, model{} 显著优于现有 RAG 基线,验证了端到端强化学习在支持复杂推理时自适应高效检索的优势。

原文摘要 · Abstract (English)

Retrieval-Augmented Generation (RAG) integrates non-parametric knowledge into Large Language Models (LLMs), typically from unstructured texts and structured graphs. While recent progress has advanced text-based RAG to multi-turn reasoning through Reinforcement Learning (RL), extending these advances to hybrid retrieval introduces additional challenges. Existing graph-based or hybrid systems typically depend on fixed or handcrafted retrieval pipelines, lacking the ability to integrate supplementary evidence as reasoning unfolds. Besides, while graph evidence provides relational structures crucial for multi-hop reasoning, it is substantially more expensive to retrieve. To address these limitations, we introduce \model{}, an RL-based framework that enables LLMs to perform multi-turn and adaptive graph-text hybrid RAG. \model{} jointly optimizes the entire generation process via RL, allowing the model to learn when to reason, what to retrieve from either texts or graphs, and when to produce final answers, all within a unified generation policy. To guide this learning process, we design a two-stage training framework that accounts for both task outcome and retrieval efficiency, enabling the model to exploit hybrid evidence while avoiding unnecessary retrieval overhead. Experimental results across five question answering benchmarks demonstrate that \model{} significantly outperforms existing RAG baselines, highlighting the benefits of end-to-end RL in supporting adaptive and efficient retrieval for complex reasoning.

检索增强强化学习多跳推理图文融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。