arXiv:2603.27752cs.CLcs.SE2026-03被引 1

提出可精确定位幻觉的层级验证框架,提升RAG系统事实一致性检测能力。

Retromorphic Testing with Hierarchical Verification for Hallucination Detection in RAG

  • 将答案拆解为可独立验证的命题,分层比对检索内容
  • 在基准数据集上实现0.776的F1值,较最强基线提升83%
  • 能定位幻觉出现的具体文本段落并提供证据支持

大语言模型在检索增强生成(RAG)中仍会产生与检索内容不符的幻觉。现有检测方法多给出整体评分,或无法提供基于证据的诊断。本文提出RT4CHART,一种面向上下文忠实性的回溯式测试框架。该框架将答案分解为独立可验证的命题,通过局部到全局的分层验证,对每个命题标注为蕴含、矛盾或无依据,并将判断结果映射回具体答案片段,返回明确的上下文证据,实现细粒度审计。在RAGTruth++(408样本)和重新标注的RAGTruth-Enhance(2,675样本)上评估显示,RT4CHART在答案级幻觉检测中取得最佳F1值。在RAGTruth++上,精度达0.845,召回率0.718,F1为0.776,较最强基线相对提升83%;在RAGTruth-Enhance上,跨度级F1达47.5%。消融实验表明,基于命题的局部处理贡献主要提升,全局验证在不同数据集上具有选择性增益。此外,重新标注发现幻觉案例数量是原标签的1.68倍,表明常用基准严重低估了幻觉发生频率。

原文摘要 · Abstract (English)

Large language models can still hallucinate in retrieval-augmented generation (RAG), producing claims that are unsupported by or conflict with the retrieved context. Detecting such errors remains challenging when faithfulness is judged solely against the retrieved context: many existing detectors return holistic answer-level scores, while others target open-domain factuality or fail to provide evidence-grounded diagnostics. We present RT4CHART, a retromorphic testing framework for context-faithfulness assessment. RT4CHART decomposes an answer into independently verifiable claims, performs hierarchical local-to-global verification against the retrieved context, and assigns each claim one of three labels: entailed, contradicted, or baseless. It further maps these claim-level decisions back to specific answer spans and returns explicit context-side evidence, enabling fine-grained auditing rather than opaque scoring. We evaluate RT4CHART on RAGTruth++ (408 samples) and our re-annotated RAGTruth-Enhance (2,675 samples). RT4CHART achieves the best answer-level hallucination-detection F1 score among the evaluated baselines. On RAGTruth++, it attains a precision of 0.845, a recall of 0.718, and an F1 score of 0.776, representing an 83% relative improvement over the strongest baseline. It also achieves a span-level F1 score of 47.5% on RAGTruth-Enhance. Ablation studies show that claim-based local processing drives most of the observed improvement, while global verification provides selective benefits across datasets. Finally, our re-annotation identifies 1.68X more hallucination cases than the original labels, suggesting that commonly used benchmarks substantially underestimate the prevalence of hallucination.

RAG幻觉检测验证框架事实性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。