用奖励模型优化检索生成，让回答更贴合人类偏好。
Reward-RAG: Enhancing RAG with Reward Driven Supervision
- 用CriticGPT训练奖励模型，指导RAG编码器学习
- 在多领域基准上表现显著优于现有方法
- 适合需要高质量生成的垂域应用
本文提出Reward-RAG，一种通过奖励驱动监督增强检索增强生成（RAG）的新方法。与以往仅关注语言模型利用外部知识不同，本方法利用CriticGPT训练专用奖励模型，将检索信息适配到特定领域，并生成合成数据用于微调RAG编码器，使其输出更符合人类偏好。该方法可通过领域微调应用于多种场景。我们在多个领域的公开基准上评估Reward-RAG，结果表明其性能显著优于当前最优方法，验证了将奖励模型与RAG结合在自然语言生成任务中具有强大潜力。
原文摘要 · Abstract (English)
In this paper, we introduce Reward-RAG, a novel approach designed to enhance the Retrieval-Augmented Generation (RAG) model through Reward-Driven Supervision. Unlike previous RAG methodologies, which focus on training language models (LMs) to utilize external knowledge retrieved from external sources, our method adapts retrieval information to specific domains by employing CriticGPT to train a dedicated reward model. This reward model generates synthesized datasets for fine-tuning the RAG encoder, aligning its outputs more closely with human preferences. The versatility of our approach allows it to be effectively applied across various domains through domain-specific fine-tuning. We evaluate Reward-RAG on publicly available benchmarks from multiple domains, comparing it to state-of-the-art methods. Our experimental results demonstrate significant improvements in performance, highlighting the effectiveness of Reward-RAG in improving the relevance and quality of generated responses. These findings underscore the potential of integrating reward models with RAG to achieve superior outcomes in natural language generation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。