用强化学习让大模型会答也会拒答，提升回答可信度
GRACE: Reinforcement Learning for Grounded Response and Abstention under Contextual Evidence
- 通过强化学习统一训练模型判断证据是否充分
- 在两个数据集上准确率领先，拒答合理率更高
- 仅需10%标注成本，适合低成本部署
检索增强生成（RAG）通过引入外部知识提升大语言模型性能，但现有系统仍存在两大缺陷：在缺乏明确证据时给出正确答案，以及在检索内容不足时生成虚假回答。以往研究分别解决这些问题，但缺乏能同时兼顾证据依赖性与可靠拒答的统一框架。本文提出GRACE，一种基于强化学习的框架，通过异构检索器自动生成多样训练样本，无需人工标注。采用多阶段门控奖励函数，训练模型评估证据充分性、提取关键证据，并在必要时明确拒绝回答。在两个基准测试上的实验表明，GRACE在整体准确率上达到当前最优水平，同时在准确回应与合理拒答之间取得良好平衡，且标注成本仅为先前方法的10%。代码已开源。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) integrates external knowledge to enhance Large Language Models (LLMs), yet systems remain susceptible to two critical flaws: providing correct answers without explicit grounded evidence and producing fabricated responses when the retrieved context is insufficient. While prior research has addressed these issues independently, a unified framework that integrates evidence-based grounding and reliable abstention is currently lacking. In this paper, we propose GRACE, a reinforcement-learning framework that simultaneously mitigates both types of flaws. GRACE employs a data construction method that utilizes heterogeneous retrievers to generate diverse training samples without manual annotation. A multi-stage gated reward function is then employed to train the model to assess evidence sufficiency, extract key supporting evidence, and provide answers or explicitly abstain. Experimental results on two benchmarks demonstrate that GRACE achieves state-of-the-art overall accuracy and strikes a favorable balance between accurate response and rejection, while requiring only 10% of the annotation costs of prior methods. Our code is available at https://github.com/YiboZhao624/Grace..
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。