解决大模型用不好外部知识的难题,提升问答准确率与事实性。
Guaranteeing Knowledge Integration with Joint Decoding for Retrieval-Augmented Generation

- 分两步生成答案:先基于内部知识推理,再强制用外部证据修正。
- 新方法使准确率最高提升12.1%,幻觉减少16.3%。
- 适合需要高可靠性的问答系统开发者使用。
检索增强生成(RAG)通过引入外部知识显著提升了大语言模型(LLMs)的表现。然而,现有研究多关注检索质量,忽视了关键的‘整合瓶颈’:即使检索到相关文档,模型仍常因与内部参数化知识冲突而无法有效利用。本文提出GuarantRAG框架,显式分离推理与证据整合过程。首先,仅基于参数化知识生成‘内答’以捕捉模型推理路径;其次,设计一种对比DPO目标生成‘引答’,将内答视为负约束,检索文档作为正样本,强制模型在该阶段抑制内部幻觉、采纳外部证据;最后,提出联合解码机制,在词元层面动态融合内答的逻辑连贯性与引答的事实精确性。在五个问答基准上的实验表明,GuarantRAG相比标准和动态RAG基线,准确率最高提升12.1%,幻觉减少16.3%。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) significantly enhances Large Language Models (LLMs) by providing access to external knowledge. However, current research primarily focuses on retrieval quality, often overlooking the critical ''integration bottleneck'': even when relevant documents are retrieved, LLMs frequently fail to utilize them effectively due to conflicts with their internal parametric knowledge. In this paper, we argue that implicitly resolving this conflict in a single generation pass is suboptimal. We introduce GuarantRAG, a framework that explicitly decouples reasoning from evidence integration. First, we generate an ''Inner-Answer'' based solely on parametric knowledge to capture the model's reasoning flow. Second, to guarantee faithful evidence extraction, we generate a ''Refer-Answer'' using a novel Contrastive DPO objective. This objective treats the parametric Inner-Answer as a negative constraint and the retrieved documents as positive ground truth, forcing the model to suppress internal hallucinations in favor of external evidence during this phase. Finally, rather than naive concatenation or using the DPO trained model directly, we propose a joint decoding mechanism that dynamically fuses the logical coherence of the Inner-Answer with the factual precision of the Refer-Answer at the token level. Experiments on five QA benchmarks demonstrate that GuarantRAG improves accuracy by up to 12.1% and reduces hallucinations by 16.3% compared to standard and dynamic RAG baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。