让大模型通过并行多查询提升推理能力，解决幻觉与延迟问题。
RAG-R1: Incentivizing the Search and Reasoning Capabilities of LLMs through Multi-query Parallelism
- 采用两阶段训练框架，实现推理过程中的多查询并行
- 在7个问答数据集上最高提升13.7%，推理时间降低11.1%
- 适合需要高可靠推理与低延迟的应用场景
大语言模型虽具备强大能力，但受限于静态内部知识，常生成幻觉或过时内容。尽管引入强化学习的检索增强生成（RAG）可缓解此问题，但其单查询模式导致推理延迟高且易出错。为此，我们提出RAG-R1，一种基于多查询并行的新型两阶段训练框架。该框架使大模型能在推理过程中自适应融合内、外部知识，并实现从单查询到多查询并行的转变。这一架构革新显著增强了推理鲁棒性，同时大幅降低推理延迟。在七个问答基准上的实验证明，该方法相较最强基线最高提升13.7%，推理时间减少11.1%。
原文摘要 · Abstract (English)
Large Language Models (LLMs), despite their remarkable capabilities, are prone to generating hallucinated or outdated content due to their static internal knowledge. While Retrieval-Augmented Generation (RAG) integrated with Reinforcement Learning (RL) offers a solution, these methods are fundamentally constrained by a single-query mode, leading to prohibitive latency and inherent brittleness. To overcome these limitations, we introduce RAG-R1, a novel two-stage training framework centered around multi-query parallelism. Our framework enables LLMs to adaptively leverage internal and external knowledge during the reasoning process while transitioning from the single-query mode to multi-query parallelism. This architectural shift bolsters reasoning robustness while significantly reducing inference latency. Extensive experiments on seven question-answering benchmarks confirm the superiority of our method, which outperforms the strongest baseline by up to 13.7% and decreases inference time by 11.1%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。