用另一模型验证大模型回答，提升幻觉检测精度与效率
Verify when Uncertain: Beyond Self-Consistency in Black Box Hallucination Detection
- 引入额外验证模型，通过跨模型一致性检测幻觉
- 在多个基准上保持高检测率，计算成本降低近半
- 适合对可靠性要求高且资源受限的应用场景
大语言模型常产生幻觉，影响其在敏感场景中的可靠性。现有黑箱设置下基于自一致性的方法表现接近有监督的黑箱最优解，提升空间有限。本文探索目标模型与额外验证模型间的跨模型一致性检测，发现结合外部信息可进一步提升检测性能。为此提出一种低成本两阶段检测算法：仅对自一致性置信度处于不确定区间的样本调用验证模型，动态切换检测策略。通过核均值嵌入的几何视角，提供一致性检测方法的理论解释。大量实验表明，该方法在问答式幻觉检测任务中维持高准确率的同时，显著降低计算开销。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often hallucinate, limiting their reliability in sensitive applications. In black-box settings, several self-consistency-based techniques have been proposed for hallucination detection. We empirically show that these methods perform nearly as well as a supervised (black-box) oracle, leaving limited room for further gains within this paradigm. To address this limitation, we explore cross-model consistency checking between the target model and an additional verifier LLM. With this extra information, we observe improved oracle performance compared to purely self-consistency-based methods. We then propose a budget-friendly, two-stage detection algorithm that calls the verifier model only for a subset of cases. It dynamically switches between self-consistency and cross-consistency based on an uncertainty interval of the self-consistency classifier. We provide a geometric interpretation of consistency-based hallucination detection methods through the lens of kernel mean embeddings, offering deeper theoretical insights. Extensive experiments on QA-style hallucination detection benchmarks show that this approach maintains high detection performance while significantly reducing computational cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。