arXiv:2510.04392cs.CLcs.AI2025-10中稿 · NeurIPS被引 2

提升检索增强系统在相似问题下的输出一致性。

Improving Consistency in Retrieval-Augmented Systems with Group Similarity Rewards

  • 通过多轮推理的组相似性奖励优化生成器,增强对语义等价查询的一致响应。
  • 在短文本、多跳和长文本问答任务中,一致性与准确率均显著优于基线模型。
  • 无需人工标注即可实现,适合高风险场景的可靠系统部署。

RAG 系统在高风险领域部署日益增多,用户期望语义等价查询产生一致输出。然而现有系统因检索器与生成器(大语言模型)的不确定性,常出现显著不一致,影响信任度。本文聚焦信息一致性,即语义等价输入应输出相同核心内容。提出一个分解式评估框架,将 RAG 一致性拆分为检索器级、生成器级与端到端三部分,帮助定位不一致根源。为此提出并行化句式组相对策略优化(PS-GRPO),利用多个改写句式集的推理结果分配组内相似性奖励。基于此构建信息一致 RAG(Con-RAG),训练生成器在改写查询间保持输出一致,并抵御检索偏差影响。由于精确计算组奖励成本过高,设计可扩展近似方法,在保持效果的同时支持大规模高效训练。跨短文本、多跳与长文本问答基准测试表明,Con-RAG 在无显式真值监督下,显著提升一致性和准确性。本工作为安全关键场景的 RAG 系统提供可落地的评估与优化方案。

原文摘要 · Abstract (English)

RAG systems are increasingly deployed in high-stakes domains where users expect outputs to be consistent across semantically equivalent queries. However, existing systems often exhibit significant inconsistencies due to variability in both the retriever and generator (LLM), undermining trust and reliability. In this work, we focus on information consistency, i.e., the requirement that outputs convey the same core content across semantically equivalent inputs. We introduce a principled evaluation framework that decomposes RAG consistency into retriever-level, generator-level, and end-to-end components, helping identify inconsistency sources. To improve consistency, we propose Paraphrased Set Group Relative Policy Optimization (PS-GRPO), an RL approach that leverages multiple rollouts across paraphrased set to assign group similarity rewards. We leverage PS-GRPO to achieve Information Consistent RAG (Con-RAG), training the generator to produce consistent outputs across paraphrased queries and remain robust to retrieval-induced variability. Because exact reward computation over paraphrase sets is computationally expensive, we also introduce a scalable approximation method that retains effectiveness while enabling efficient, large-scale training. Empirical evaluations across short-form, multi-hop, and long-form QA benchmarks demonstrate that Con-RAG significantly improves both consistency and accuracy over strong baselines, even in the absence of explicit ground-truth supervision. Our work provides practical solutions for evaluating and building reliable RAG systems for safety-critical deployments.

RAG一致性强化学习生成优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。