提出可验证的证据充分性检查方法，提升RAG回答可靠性。
SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generation
- 将证据充分性视为整体属性，通过聚合多个证据关系判断答案支持度。
- 在HotpotQA-RAG上达到0.9075宏F1，比GPT-4o高13%以上。
- 适合需要可审计、低幻觉风险的高可靠问答系统使用。
检索增强生成（RAG）虽能将答案锚定在检索段落中，但相关性不等于充分性：一个主题相关的段落仍可能不足以支撑答案。本文研究选择性RAG中的证据充分性验证问题，即验证器接收问题、候选答案和检索证据后，判断证据是否支持、反驳或不足以支持答案，并仅在有支持时才输出答案。提出SURE-RAG聚合协议，将证据充分性视为集合级属性，独立评分无法发现缺失推理链或未解决矛盾。共享的主张-证据验证器为每对（主张，段落）生成局部关系分布，再由SURE-RAG聚合为四个可解释的答案级特征块（覆盖度、关系强度、不确定性、检索质量），实现三分类决策与可审计的选答分数。在控制型多跳基准HotpotQA-RAG v3上评估，采用抗干扰协议（捷径基线、反事实替换、无真值检测与GPT-4o审计）。校准后的SURE-RAG取得0.9075宏F1（原始0.8951 ± 0.0069），显著优于DeBERTa平均池化（0.6516）和GPT-4o裁判（0.7284），接近强交叉编码器（0.8888 ± 0.0109）水平，且全程可审计。当覆盖度为30%时，风险从0.2588降至0.1642，相对下降37%。边界实验对比其与GPT-4o在HaluBench上的幻觉检测表现，排名反转（0.3343 vs. 0.7389 unsafe-F1），表明受控充分性验证与自然幻觉检测是不同问题。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) grounds answers in retrieved passages, yet relevance does not guarantee sufficiency: a topical passage may still fail to justify the answer. We study evidence sufficiency verification for selective RAG answering, in which a verifier receives a question, a candidate answer, and retrieved evidence and decides whether the evidence supports, refutes, or is insufficient for the answer, answering only when support is established. We present SURE-RAG, an aggregation protocol that treats evidence sufficiency as a set-level property: missing hops and unresolved conflicts cannot be detected by scoring passages independently. A shared claim-evidence verifier produces a local relation distribution for each (claim, passage) pair, which SURE-RAG aggregates into four interpretable answer-level feature blocks (coverage, relation strength, uncertainty, and retrieval), producing a three-way decision and an auditable selective score. We evaluate on HotpotQA-RAG v3, a controlled multi-hop benchmark, under an artifact-aware protocol (shortcut baselines, counterfactual swaps, no-oracle checks, and GPT-4o audits). Calibrated SURE-RAG attains 0.9075 Macro-F1 (raw 0.8951 +/- 0.0069), well above DeBERTa mean-pooling (0.6516) and a GPT-4o judge (0.7284), and on par with a strong concat cross-encoder (0.8888 +/- 0.0109) while remaining fully auditable. At 30% coverage, risk falls from 0.2588 to 0.1642, a 37% relative reduction. As a boundary-mapping experiment, we contrast SURE-RAG with GPT-4o on HaluBench unsafe detection: the ranking reverses (0.3343 vs. 0.7389 unsafe-F1), indicating that controlled sufficiency verification and natural hallucination detection are distinct problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。