用互动证明框架让大模型学会根据证据判断是否回答,减少幻觉。
Bounding Hallucinations: Merlin-Arthur Protocols for Mutual-Information Bounds in Language Models
- 将检索增强生成设计为梅林-亚瑟互动证明系统,让模型自检证据可信度。
- 在五个问答数据集上,错误回答率下降18-35个百分点,无需人工标注无答案样本。
- 提出解释信息占比新指标,首次量化检索内容对答案的信息贡献度。
检索增强生成(RAG)依赖检索到的上下文引导大语言模型,但将检索视为启发式而非可验证证据,导致生成不支持的答案、出现幻觉并依赖虚假上下文。本文提出一种新型评估、数据增强与训练框架,将RAG流程建模为梅林-亚瑟(M/A)协议:亚瑟(生成模型)接收未知来源的上下文,梅林提供有帮助的证据,摩根娜则注入对抗性误导上下文。通过XAI方法实现对影响亚瑟决策的关键证据的自我评估与修正。基于此框架,提出解释信息占比(EIF)评分,分离解释保真度与模型预测误差,且将互信息下界归一化至真实场景。训练后,亚瑟学会在证据充分时回答,在不足时拒绝回答。在五个QA基准和四个模型(1B至32B参数)上,训练期间错误率降低最高达35个百分点,比基线微调提升18-20个百分点。无需人工标注无答案样本即可自发产生拒答行为。训练中EIF得分提升0.1-0.4,相比基线提升0.33-0.38。复用梅林/摩根娜上下文作为自动难例,使检索器Recall@1提升2个百分点。尽管高准确率未必代表上下文到答案的信息传递,但我们的EIF分数——据我们所知是首个针对RAG生成器的上下文到答案信息边界——表明自主互动证明式监督可使RAG系统将检索文档视作可验证证据。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) relies on retrieved context to guide large language models (LLM), yet treats the retrieval as a heuristic rather than verifiable evidence -- leading to unsupported answers, hallucinations, and reliance on spurious context. We introduce a novel evaluation, data augmentation, and training framework that treats the RAG pipeline as an interactive proof system by adapting the Merlin-Arthur (M/A) protocol: Arthur (the generator LLM) receives context of unknown provenance and Merlin gives helpful evidence, while Morgana injects adversarial, misleading context. We implement them both with an XAI method to self-assess and modify evidence most influential to Arthur. Based on this framework we propose the Explained Information Fraction (EIF) score, that disentangles explanation fidelity from model predictive errors and imperfect benchmarks, and normalizes M/A mutual-information lower bounds to realistic empirical settings. When trained with those contexts, Arthur learns to answer when evidence supports the answer and abstain when evidence is insufficient. Across five QA benchmarks and four LLMs (1B to 32B parameters), M/A reduces incorrect answers under insufficient context by up to 35pp during training and by 18-20pp over vanilla finetuning. Abstention emerges even \emph{without any manually annotated unanswerable example or preference pair}. We improve EIF-cond by 0.1-0.4 during M/A training and by 0.33-0.38 over vanilla finetuning. Reusing Merlin/Morgana contexts as automatic hard positives and negatives also raises retriever Recall@1 by 2pp. While high accuracy does not guarantee entropy flow from context to answer, our EIF scores -- to our knowledge, the first context-to-answer information bound for RAG generators -- show that autonomous interactive-proof-style supervision enables RAG systems that treat retrieved documents as verifiable evidence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。