通过生成证据笔记提升RAG模型推理准确率
EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notes
- 先生成凝练的支撑性证据笔记,再基于笔记推理
- 在HotpotQA等数据集上相对F1提升最高达91%
- 适合需要高精度推理的开放域问答场景
检索增强生成(RAG)通过引入外部信息提升开放域问答性能,但仍面临两大挑战:一是外部信息中有效信号被无关内容稀释,二是多跳推理中因不完整或错误信息导致错误累积。为此,我们提出EviNote-RAG框架,采用检索-笔记-回答的工作流。模型首先生成支撑性证据笔记(SENs),精炼保留关键答案信息并标注重要性与不确定性,以提升准确性;进一步设计基于蕴含关系的证据质量奖励(EQR),确保笔记逻辑上足以推导出最终答案。在领域内与跨域问答基准上的实验表明,EviNote-RAG达到当前最优性能,在答案准确率、训练稳定性、鲁棒性和效率方面均有提升。具体而言,在HotpotQA上相对F1提升20%(+0.093),Bamboogle上提升40%(+0.151),2Wiki上提升91%(+0.256),显著优化了推理过程。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has advanced open-domain question answering by incorporating external information into model reasoning. However, effectively leveraging external information to enhance reasoning presents the following challenges: (1) low signal-to-noise ratio, where answer-supportive external information is diluted by irrelevant material, and (2) error accumulation, which arises in multi-hop reasoning when incomplete or misleading information is incorporated. To address these challenges, we introduce EviNote-RAG, a framework that follows a retrieve-note-answer workflow. Instead of reasoning directly over raw external information, the model first produces Supportive-Evidence Notes (SENs), which concisely preserve answer-critical information and explicitly mark key and uncertainty information to improve accuracy. We further design an entailment-based Evidence Quality Reward (EQR) to ensure that SENs are logically sufficient to derive the final answer, thereby enhancing SENs' quality. Experiments on both in-domain and out-of-domain QA benchmarks show that EviNote-RAG achieves state-of-the-art performance, improving answer accuracy, training stability, robustness, and efficiency. In particular, it yields relative F1 gains of 20% on HotpotQA (+0.093), 40% on Bamboogle (+0.151), and 91% on 2Wiki (+0.256), benefiting from improvements in the reasoning process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。