仅用概率就能估计大模型不确定性，无需额外计算。
Probabilities Are All You Need: A Probability-Only Approach to Uncertainty Estimation in Large Language Models
- 用输出概率的前K项近似预测熵，无需多次采样。
- 在多个大模型和问答数据集上优于现有方法。
- 适合需要快速评估模型可信度的研究与应用。
大型语言模型（LLMs）在自然语言处理任务中表现优异，但易产生幻觉，生成事实错误或误导性内容。不确定性估计是缓解该问题的关键，通常依赖预测熵。然而，现有方法常需多次采样或额外计算来评估语义熵。本文提出一种高效、无需训练的不确定性估计方法，通过响应结果的前K个概率近似预测熵，并采用自适应机制动态确定K值，以增强灵活性并过滤低置信度概率。在三个自由形式问答数据集上，针对多个LLM的实验表明，该方法优于昂贵的先进基线，有助于提升大模型的可信度。
原文摘要 · Abstract (English)
Large Language Models (LLMs) exhibit strong performance across various natural language processing (NLP) tasks but remain vulnerable to hallucinations, generating factually incorrect or misleading outputs. Uncertainty estimation, often using predictive entropy estimation, is key to addressing this issue. However, existing methods often require multiple samples or extra computation to assess semantic entropy. This paper proposes an efficient, training-free uncertainty estimation method that approximates predictive entropy using the responses' top-$K$ probabilities. Moreover, we employ an adaptive mechanism to determine $K$ to enhance flexibility and filter out low-confidence probabilities. Experimental results on three free-form question-answering datasets across several LLMs demonstrate that our method outperforms expensive state-of-the-art baselines, contributing to the broader goal of enhancing LLM trustworthiness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。