让大模型更懂检索内容的可靠性，提升问答准确率。
RPO: Retrieval Preference Optimization for Robust Retrieval-Augmented Generation
- 用隐式相关性信号优化生成过程，统一评估与生成
- 在4个数据集上比传统RAG高4%-10%准确率
- 无需额外模块，适合追求稳定性能的部署场景
尽管检索增强生成（RAG）在利用外部知识方面展现出潜力，但其生成过程高度依赖检索内容的质量与准确性。大型语言模型难以判断外部非参数化知识与内部记忆不一致时的正确性，导致生成过程中出现知识冲突。为此，我们提出检索偏好优化（RPO），一种轻量且有效的对齐方法，可基于检索相关性自适应地利用多源知识。通过提取检索相关性的隐式表示并融入奖励模型，将检索评估与响应生成整合至单一模型中，解决了以往方法需额外步骤评估检索质量的问题。值得注意的是，RPO是唯一在训练中量化检索相关性意识的RAG专用对齐方法，克服了数学障碍。在四个数据集上的实验表明，RPO在不引入任何额外组件的情况下，准确率比RAG高出4%-10%，展现出强大的泛化能力。
原文摘要 · Abstract (English)
While Retrieval-Augmented Generation (RAG) has exhibited promise in utilizing external knowledge, its generation process heavily depends on the quality and accuracy of the retrieved context. Large language models (LLMs) struggle to evaluate the correctness of non-parametric knowledge retrieved externally when it differs from internal memorization, leading to knowledge conflicts during response generation. To this end, we introduce the Retrieval Preference Optimization (RPO), a lightweight and effective alignment method to adaptively leverage multi-source knowledge based on retrieval relevance. An implicit representation of retrieval relevance is derived and incorporated into the reward model to integrate retrieval evaluation and response generation into a single model, solving the problem that previous methods necessitate the additional procedure to assess the retrieval quality. Notably, RPO is the only RAG-dedicated alignment approach that quantifies the awareness of retrieval relevance in training, overcoming mathematical obstacles. Experiments on four datasets demonstrate that RPO outperforms RAG by 4-10% in accuracy without any extra component, exhibiting its robust generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。