揭示RAG幻觉检测的语义幻觉陷阱，证明仅靠嵌入相似性无法可靠识别真实幻觉。
The Semantic Illusion: Certified Limits of Embedding-Based Hallucination Detection in RAG Systems
- 用置信预测将幻觉检测转为带有限样本覆盖率保证的决策集。
- 在真实幻觉数据上嵌入方法误报率达100%，而合成数据上仅为0%。
- 指出语义上难以区分幻觉与真实回答，适合关注安全性的研究者阅读。
检索增强生成（RAG）系统虽基于检索证据，仍易产生幻觉。现有检测方法依赖嵌入相似性和自然语言推理（NLI），但其在安全关键场景下的可靠性未被验证。本文采用置信预测，将启发式得分转化为具有有限样本覆盖保证（1-α）的决策集。基于n=600的校准集，我们发现根本性矛盾：在合成幻觉（Natural Questions）上，嵌入方法实现95%覆盖率且0%假阳性率（FPR）；但在真实幻觉数据（来自RLHF对齐模型的HaluEval）上，相同方法失败，目标覆盖率下假阳性率达100%。通过分布尾部分析，我们发现尽管NLI模型的AUC为0.81，最困难的幻觉在语义上与真实回复几乎无法区分，迫使置信阈值拒绝几乎所有有效输出。关键的是，GPT-4作为判官在相同数据上实现7% FPR（95% CI: [3.4%, 13.7%]），证明该任务可通过推理解决，却对表层语义不可见——这一现象被称为“语义幻觉”。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems remain susceptible to hallucinations despite grounding in retrieved evidence. While current detection methods leverage embedding similarity and natural language inference (NLI), their reliability in safety-critical settings remains unproven. We apply conformal prediction to RAG hallucination detection, transforming heuristic scores into decision sets with finite-sample coverage guarantees (1-alpha). Using calibration sets of n=600, we demonstrate a fundamental dichotomy: on synthetic hallucinations (Natural Questions), embedding methods achieve 95% coverage with 0% False Positive Rate (FPR). However, on real hallucinations from RLHF-aligned models (HaluEval), the same methods fail catastrophically, yielding 100% FPR at target coverage. We analyze this failure through the lens of distributional tails, showing that while NLI models achieve acceptable AUC (0.81), the "hardest" hallucinations are semantically indistinguishable from faithful responses, forcing conformal thresholds to reject nearly all valid outputs. Crucially, GPT-4 as a judge achieves 7% FPR (95% CI:[3.4%, 13.7%]) on the same data, proving the task is solvable via reasoning but opaque to surface-level semantics--a phenomenon we term the "Semantic Illusion."
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。