用增益信号对齐检索器与大模型偏好,提升生成准确性。
GainRAG: Preference Alignment in Retrieval-Augmented Generation through Gain Signal Synthesis
- 定义'增益'度量片段对正确输出的贡献,指导检索优化。
- 仅需少量数据训练中间模块,有效对齐检索与生成偏好。
- 引入伪片段策略缓解性能下降,6个数据集验证有效。
检索增强生成(RAG)框架通过动态注入检索内容来提升大语言模型(LLMs)的表现,但在实际应用中存在检索器与大模型间的偏好差异,制约了系统进一步优化。部分高度相关的段落因信息复杂或矛盾反而干扰推理;而间接相关或错误内容可能提供有益提示或逻辑线索,帮助生成更准确答案。为此,我们提出GainRAG,通过定义新指标'增益'来衡量输入段落对正确输出的贡献。具体地,我们设计方法估计增益信号,并利用有限数据训练一个中间模块,以对齐检索器与大模型的偏好。此外,引入伪片段策略缓解性能退化。在6个数据集上的实验结果验证了GainRAG的有效性。
原文摘要 · Abstract (English)
The Retrieval-Augmented Generation (RAG) framework introduces a retrieval module to dynamically inject retrieved information into the input context of large language models (LLMs), and has demonstrated significant success in various NLP tasks. However, the current study points out that there is a preference gap between retrievers and LLMs in the RAG framework, which limit the further improvement of system performance. Some highly relevant passages may interfere with LLM reasoning because they contain complex or contradictory information; while some indirectly related or even inaccurate content may help LLM generate more accurate answers by providing suggestive information or logical clues. To solve this, we propose GainRAG, a novel approach that aligns the retriever's and LLM's preferences by defining a new metric, "gain", which measure how well an input passage contributes to correct outputs. Specifically, we propose a method to estimate these gain signals and train a middleware that aligns the preferences of the retriever and the LLM using only limited data. In addition, we introduce a pseudo-passage strategy to mitigate degradation. The experimental results on 6 datasets verify the effectiveness of GainRAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。