让大模型学会在错误信息中坚持正确知识，提升推理可靠性。
Resisting Contextual Interference in RAG via Parametric-Knowledge Reinforcement
- 用强化学习训练模型优先依赖自身知识，而非被误导性检索内容干扰。
- 在虚构场景下比顶尖方法准确率高22.89%，且在正确检索时无性能损失。
- 适合需要高可靠性的问答、医疗和法律等知识密集型任务使用。
检索增强生成（RAG）虽能提升知识密集型任务表现，但易受错误、无关或冲突的检索文本干扰，导致模型依赖不准确证据并引发错误级联。我们提出Knowledgeable-R1，一种基于强化学习的框架，显式训练大语言模型利用参数化知识（PK）抵抗上下文干扰，同时在外部上下文可信时仍能有效利用。该方法引入联合采样机制，生成有无检索条件下的成对响应，并在相同输入下学习局部优势（每种解码策略内）与全局优势，以量化何时应忽略误导性上下文或采纳它。采用非对称优势变换，强化向参数化知识探索的行为。实验表明，Knowledgeable-R1显著提升知识冲突与一般RAG场景下的鲁棒性与推理准确性，在反事实场景中优于最先进基线+22.89%，且当检索内容完全准确时无性能下降。代码已开源：https://github.com/lcy80366872/knowledgeable-R1。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) improves performance on knowledge-intensive tasks but can be derailed by wrong, irrelevant, or conflicting retrieved text, causing models to rely on inaccurate evidence and cascade errors. We propose Knowledgeable-R1, a reinforcement-learning framework that explicitly trains large language models to use parametric knowledge (PK) to resist contextual interference while still exploiting external context when it is reliably helpful. Knowledgeable-R1 introduces a joint sampling scheme that generates paired responses with and without retrieval, and learns both local advantages (within each decoding regime) and global advantages under the same input to quantify when to ignore misleading context versus adopt it. We employ an asymmetric advantage transformation that amplifies exploratory behaviors toward parametric knowledge. Experiments show that Knowledgeable-R1 significantly improves robustness and reasoning accuracy in knowledge conflict scenarios and general RAG scenarios, outperforming SOTA baselines by +22.89% in counterfactual scenarios, and without degradation when the retrieved context is fully accurate.Our code are available at https://github.com/lcy80366872/knowledgeable-R1.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。