arXiv:2504.14858cs.AIcs.CL2025-04中稿 · NeurIPS被引 3

让大模型推理更可信,通过自动纠错提升检索增强生成的准确性

Retrieval is Not Enough: Enhancing RAG Reasoning through Test-Time Critique and Optimization

  • 用批判性反馈机制动态校正模型推理过程
  • 在跨领域任务上比基线提升12.1%,超越72B大模型2.2%
  • 自动决定优化次数,无需预设迭代次数,适合实际部署

检索增强生成(RAG)已成为知识驱动大模型的主流范式。但标准RAG常导致模型推理与检索证据不一致,产生事实错误或无依据结论。本文将RAG重新定义为检索增强推理,识别出核心问题:推理错位——即模型内部推理轨迹与检索证据约束之间的偏离。为此提出AlignRAG,一种基于批判驱动对齐(CDA)的迭代框架,并引入可自主终止的AlignRAG-auto版本,无需预设迭代次数。其核心是对比批判合成机制,生成依赖检索结果的批判性反馈,同时抑制自我偏见。该机制训练专用的检索增强批判语言模型(CLM),使用标注数据区分符合证据与偏离证据的推理。实验表明,8B参数的CLM在跨域任务上比Self-Refine基线提升12.1%,优于标准72B参数CLM 2.2%。AlignRAG-auto实现当前最优性能的同时,动态确定最佳优化步数,提升效率与可用性。该方法兼容现有RAG架构,作为即插即用模块,在信息丰富和噪声检索场景下均表现稳健。

原文摘要 · Abstract (English)

Retrieval-augmented generation (RAG) has become a widely adopted paradigm for enabling knowledge-grounded large language models (LLMs). However, standard RAG pipelines often fail to ensure that model reasoning remains consistent with the evidence retrieved, leading to factual inconsistencies or unsupported conclusions. In this work, we reinterpret RAG as Retrieval-Augmented Reasoning and identify a central but underexplored problem: Reasoning Misalignment -- the divergence between an LLM's internal reasoning trajectory and the evidential constraints provided by retrieval. To address this issue, we propose AlignRAG, a novel iterative framework grounded in Critique-Driven Alignment (CDA). We further introduce AlignRAG-auto, an autonomous variant that dynamically terminates refinement, removing the need to pre-specify the number of critique iterations. At the heart of AlignRAG lies a contrastive critique synthesis mechanism that generates retrieval-sensitive critiques while mitigating self-bias. This mechanism trains a dedicated retrieval-augmented Critic Language Model (CLM) using labeled critiques that distinguish between evidence-aligned and misaligned reasoning. Empirical evaluations show that our approach significantly improves reasoning fidelity. Our 8B-parameter CLM improves performance over the Self-Refine baseline by 12.1% on out-of-domain tasks and outperforms a standard 72B-parameter CLM by 2.2%. Furthermore, AlignRAG-auto achieves this state-of-the-art performance while dynamically determining the optimal number of refinement steps, enhancing efficiency and usability. AlignRAG remains compatible with existing RAG architectures as a plug-and-play module and demonstrates strong robustness under both informative and noisy retrieval scenarios.

RAG推理对齐自动优化批判模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。