arXiv:2604.19162cs.CLstat.AP2026-04

通过融合语义覆盖率与图谱信息,提升小样本下大模型幻觉检测的准确性。

Mind the Unseen Mass: Unmasking LLM Hallucinations via Soft-Hybrid Alphabet Estimation

论文配图:Mind the Unseen Mass: Unmasking LLM Hallucinations via Soft-Hybrid Alphabet Estimation
图 1 · 摘自论文原文
  • 结合广义古德图灵法与蕴含加权图的热核迹,动态融合两种信号。
  • 在小样本条件下,对语义模式缺失的检测准确率提升显著,优于传统方法。
  • 适合资源受限场景中大模型不确定性评估,如API调用受限时的风险控制。

本文研究在仅能获取少量响应样本的黑箱环境下,大型语言模型(LLMs)的不确定性量化问题。估计有效语义字母表规模——即采样响应中表达的不同语义数量——可作为下游风险的有用代理指标。然而,基于频率的估计器在小样本时易低估稀有语义模式,而仅依赖图谱谱量无法精确估计语义占据率。为此,我们提出SHADE(Soft-Hybrid Alphabet Dynamic Estimator),一种简单且可解释的估计器,融合广义古德图灵覆盖率与从蕴含加权图构建的归一化拉普拉斯矩阵的热核迹。估计的覆盖率自适应决定融合规则:高覆盖率时采用凸组合,低覆盖率时使用LogSumExp融合以强调缺失或弱观测语义模式。随后引入有限样本修正以稳定基数估计,并转换为覆盖调整后的语义熵得分。在聚合语义字母表规模估计与问答错误检测任务上,SHADE在样本最有限的情况下表现最优,随样本增加性能差距缩小。结果表明,混合语义占据估计在采样预算紧张时尤为有益。

原文摘要 · Abstract (English)

This paper studies uncertainty quantification for large language models (LLMs) under black-box access, where only a small number of responses can be sampled for each query. In this setting, estimating the effective semantic alphabet size--that is, the number of distinct meanings expressed in the sampled responses--provides a useful proxy for downstream risk. However, frequency-based estimators tend to undercount rare semantic modes when the sample size is small, while graph-spectral quantities alone are not designed to estimate semantic occupancy accurately. To address this issue, we propose SHADE (Soft-Hybrid Alphabet Dynamic Estimator), a simple and interpretable estimator that combines Generalized Good-Turing coverage with a heat-kernel trace of the normalized Laplacian constructed from an entailment-weighted graph over sampled responses. The estimated coverage adaptively determines the fusion rule: under high coverage, SHADE uses a convex combination of the two signals, while under low coverage it applies a LogSumExp fusion to emphasize missing or weakly observed semantic modes. A finite-sample correction is then introduced to stabilize the resulting cardinality estimate before converting it into a coverage-adjusted semantic entropy score. Experiments on pooled semantic alphabet-size estimation against large-sample references and on QA incorrectness detection show that SHADE achieves the strongest improvements in the most sample-limited regime, while the performance gap narrows as the number of samples increases. These results suggest that hybrid semantic occupancy estimation is particularly beneficial when black-box uncertainty quantification must operate under tight sampling budgets.

大模型不确定性幻觉检测小样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。