arXiv:2601.19918cs.CL2026-01被引 3

仅需一次推理,就能高效检测大模型幻觉。

Lowest Span Confidence: A Zero-Shot Metric for Efficient and Black-Box Hallucination Detection in LLMs

  • 用滑动窗口分析语义连贯片段的联合置信度。
  • 在多个模型和基准上优于现有零样本方法。
  • 适合资源受限的黑盒API场景使用。

大语言模型生成看似合理但非事实内容的幻觉问题,严重制约其在高风险场景中的可靠应用。现有检测方法通常依赖昂贵的采样策略或白盒模型状态,难以适用于常见的API调用场景。为此,本文提出一种新型高效零样本指标——最低段落置信度(LSC),仅需单次前向传播及输出概率即可实现幻觉检测。LSC通过滑动窗口机制评估语义连贯片段的联合似然,识别不同长度n-gram中最低的边缘置信度区域,有效捕捉与事实不一致强相关的局部不确定性模式。相比困惑度的稀释效应和最小词元概率的噪声敏感性,LSC提供更鲁棒的事实不确定性估计。在多个SOTA模型和多样化基准上的实验表明,LSC始终超越现有零样本基线,在资源受限条件下仍保持优异检测性能。

原文摘要 · Abstract (English)

Hallucinations in Large Language Models (LLMs), i.e., the tendency to generate plausible but non-factual content, pose a significant challenge for their reliable deployment in high-stakes environments. However, existing hallucination detection methods generally operate under unrealistic assumptions, i.e., either requiring expensive intensive sampling strategies for consistency checks or white-box LLM states, which are unavailable or inefficient in common API-based scenarios. To this end, we propose a novel efficient zero-shot metric called Lowest Span Confidence (LSC) for hallucination detection under minimal resource assumptions, only requiring a single forward with output probabilities. Concretely, LSC evaluates the joint likelihood of semantically coherent spans via a sliding window mechanism. By identifying regions of lowest marginal confidence across variable-length n-grams, LSC could well capture local uncertainty patterns strongly correlated with factual inconsistency. Importantly, LSC can mitigate the dilution effect of perplexity and the noise sensitivity of minimum token probability, offering a more robust estimate of factual uncertainty. Extensive experiments across multiple state-of-the-art (SOTA) LLMs and diverse benchmarks show that LSC consistently outperforms existing zero-shot baselines, delivering strong detection performance even under resource-constrained conditions.

幻觉检测零样本置信度大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。