通过语义结构熵，实现大模型不确定性量化，避免幻觉。
SeSE: Black-Box Uncertainty Quantification for Large Language Models Based on Structural Information Theory
- 基于编码树构建语义结构熵，捕捉深层语义信息。
- 在24组模型-数据组合上优于现有方法，支持长文本输出。
- 适用于闭源/开源模型，结果可解释，适合安全场景。
可靠的不确定性量化（UQ）对大语言模型（LLMs）在安全关键场景中的部署至关重要，可使模型在不确定时拒绝回答，从而避免幻觉——即看似合理但事实错误的回应。然而，现有语义UQ方法忽略了潜在的语义结构信息，可能影响估计精度。本文提出一种基于结构信息理论的黑盒UQ框架——语义结构熵（SeSE），适用于开箱与闭源模型。SeSE通过最小化结构熵构建最优层次抽象编码树，以揭示语义空间的内在结构。该编码树的结构熵即量化了经最优压缩后的模型语义空间中的固有不确定性。此外，不同于仅关注短文本生成的方法，本工作将SeSE扩展至长文本输出，提供可解释、细粒度的不确定性估计。理论上证明了SeSE是语义熵（当前UQ黄金标准）的推广，实证表明其在24个模型-数据组合中均显著优于强基线。
原文摘要 · Abstract (English)
Reliable uncertainty quantification (UQ) is essential for deploying large language models (LLMs) in safety-critical scenarios, as it enables them to abstain from responding when uncertain, thereby avoiding hallucinations, i.e., plausible yet factually incorrect responses. However, while semantic UQ methods have achieved advanced performance, they overlook latent semantic structural information that could enable more precise uncertainty estimates. In this paper, we propose \underline{Se}mantic \underline{S}tructural \underline{E}ntropy ({SeSE}), a principled black-box UQ framework applicable to both open- and closed-source LLMs. To reveal the intrinsic structure of the semantic space, SeSE constructs its optimal hierarchical abstraction through an encoding tree with minimal structural entropy. The structural entropy of this encoding tree thus quantifies the inherent uncertainty within LLM semantic space after optimal compression. Additionally, unlike existing methods that primarily focus on simple short-form generation, we extent SeSE to provide interpretable, granular uncertainty estimation for long-form outputs. We theoretically prove that SeSE generalizes semantic entropy, the gold standard for UQ in LLMs, and empirically demonstrate its superior performance over strong baselines across 24 model-dataset combinations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。