用强化学习优化检索，让专业领域问答更准更快
Domain-Aware RAG: MoL-Enhanced RL for Efficient Training and Scalable Retrieval
- 用多损失持续预训练+强化学习联合优化查询和文档
- 在多个基准数据集上召回率显著超越现有方法
- 适合需要高效精准检索的垂直领域应用
检索增强生成（RAG）系统高度依赖检索阶段，尤其是粗排过程。现有粗排优化方法常难以平衡领域知识学习与查询增强，导致检索性能不佳。为此，我们提出MoLER，一种面向领域的RAG方法，通过摩尔增强的强化学习优化检索。MoLER采用两阶段流程：第一阶段使用多损失（MoL）进行持续预训练（CPT），平衡领域知识与通用语言能力；第二阶段采用组相对策略优化（GRPO）强化学习，优化查询与段落生成以最大化文档召回率。关键创新在于多查询单段落延迟融合（MSLF）策略，在强化学习训练中降低计算开销，同时通过多查询多段落延迟融合（MMLF）实现可扩展推理。在多个基准数据集上的大量实验表明，MoLER达到当前最优性能，显著优于基线方法。该方法弥合了RAG系统中的知识鸿沟，支持专业化领域的鲁棒且可扩展的检索。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems rely heavily on the retrieval stage, particularly the coarse-ranking process. Existing coarse-ranking optimization approaches often struggle to balance domain-specific knowledge learning with query enhencement, resulting in suboptimal retrieval performance. To address this challenge, we propose MoLER, a domain-aware RAG method that uses MoL-Enhanced Reinforcement Learning to optimize retrieval. MoLER has a two-stage pipeline: a continual pre-training (CPT) phase using a Mixture of Losses (MoL) to balance domain-specific knowledge with general language capabilities, and a reinforcement learning (RL) phase leveraging Group Relative Policy Optimization (GRPO) to optimize query and passage generation for maximizing document recall. A key innovation is our Multi-query Single-passage Late Fusion (MSLF) strategy, which reduces computational overhead during RL training while maintaining scalable inference via Multi-query Multi-passage Late Fusion (MMLF). Extensive experiments on benchmark datasets show that MoLER achieves state-of-the-art performance, significantly outperforming baseline methods. MoLER bridges the knowledge gap in RAG systems, enabling robust and scalable retrieval in specialized domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。