轻量级工具可快速检测RAG生成内容的虚构信息。
Provenance: A Light-weight Fact-checker for Retrieval Augmented LLM Generation Output
- 用开源自然语言推理模型计算事实性得分，无需微调大模型。
- 在多个开源数据集上AUC高，能有效识别生成内容中的幻觉。
- 可定位幻觉来源的上下文片段，适合需要可信输出的场景。
我们提出一种轻量级方法，用于检测检索增强生成（RAG）系统输出中的非事实内容。给定上下文和待检测输出，该方法计算一个事实性评分，通过阈值判断是否需验证LLM问答、摘要等系统的输出。不同于依赖大模型的事实核查器，本方法采用小型开源自然语言推理（NLI）模型，实现低延迟、低成本的实时检测，且无需对LLM进行微调。该方法还可追踪幻觉来源至具体上下文块，支持后续修正。实验表明，在多个相关开源数据集上均取得高受试者工作特征曲线下面积（AUC），证明其在检测RAG输出幻觉方面的有效性。
原文摘要 · Abstract (English)
We present a light-weight approach for detecting nonfactual outputs from retrieval-augmented generation (RAG). Given a context and putative output, we compute a factuality score that can be thresholded to yield a binary decision to check the results of LLM-based question-answering, summarization, or other systems. Unlike factuality checkers that themselves rely on LLMs, we use compact, open-source natural language inference (NLI) models that yield a freely accessible solution with low latency and low cost at run-time, and no need for LLM fine-tuning. The approach also enables downstream mitigation and correction of hallucinations, by tracing them back to specific context chunks. Our experiments show high area under the ROC curve (AUC) across a wide range of relevant open source datasets, indicating the effectiveness of our method for fact-checking RAG output.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。