让大模型学会用多种检索工具动态找信息,提升推理准确性
MARAG-R1: Beyond Single Retriever via Reinforcement-Learned Multi-Tool Agentic Retrieval
- 用四个检索工具+强化学习训练,动态选择最佳检索方式
- 在多个数据集上超越现有方法,最高提升6.8个百分点
- 适合需要跨文档推理和精准信息获取的复杂问答场景
大型语言模型(LLMs)在推理和生成方面表现优异,但受限于静态预训练数据,常出现事实性错误且难以适应新信息。检索增强生成(RAG)通过引入外部知识缓解此问题,但其效果受限于能否有效获取相关知识。现有RAG系统依赖单一检索器进行固定top-k选择,导致只能访问语料库中狭窄且静态的子集,成为全面获取外部信息的主要瓶颈,尤其在需要全库推理的任务中。为此,我们提出MARAG-R1,一种基于强化学习的多工具智能体检索框架,使大模型能动态协调多种检索机制,实现更广泛、更精确的信息获取。MARAG-R1赋予模型四种检索工具:语义搜索、关键词搜索、过滤与聚合,并通过两阶段训练(监督微调+强化学习)学习何时及如何使用这些工具。该设计支持推理与检索的交错进行,逐步收集足够证据以完成全库级综合。在GlobalQA、HotpotQA和2WikiMultiHopQA上的实验表明,MARAG-R1显著优于强基线,实现了新的最先进水平。
原文摘要 · Abstract (English)
Large Language Models (LLMs) excel at reasoning and generation but are inherently limited by static pretraining data, resulting in factual inaccuracies and weak adaptability to new information. Retrieval-Augmented Generation (RAG) addresses this issue by grounding LLMs in external knowledge; However, the effectiveness of RAG critically depends on whether the model can adequately access relevant information. Existing RAG systems rely on a single retriever with fixed top-k selection, restricting access to a narrow and static subset of the corpus. As a result, this single-retriever paradigm has become the primary bottleneck for comprehensive external information acquisition, especially in tasks requiring corpus-level reasoning. To overcome this limitation, we propose MARAG-R1, a reinforcement-learned multi-tool RAG framework that enables LLMs to dynamically coordinate multiple retrieval mechanisms for broader and more precise information access. MARAG-R1 equips the model with four retrieval tools -- semantic search, keyword search, filtering, and aggregation -- and learns both how and when to use them through a two-stage training process: supervised fine-tuning followed by reinforcement learning. This design allows the model to interleave reasoning and retrieval, progressively gathering sufficient evidence for corpus-level synthesis. Experiments on GlobalQA, HotpotQA, and 2WikiMultiHopQA demonstrate that MARAG-R1 substantially outperforms strong baselines and achieves new state-of-the-art results in corpus-level reasoning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。