让大模型自己评估回答可信度，只需两样本就有效。
Clustered Self-Assessment: A Simple yet Effective Method for Uncertainty Quantification in Large Language Models
- 将生成结果聚类成选项，让模型自评每项信心。
- 仅用2个样本即达良好表现，优于传统方法。
- 适合需要可靠输出的场景，如医疗与法律。
大型语言模型在各类任务中表现优异，但常生成看似合理实则错误的回答。缺乏明确的不确定性估计使用户难以判断输出可靠性。现有方法多依赖采样生成的熵等间接信号，难以解释且未充分利用模型自评估能力。本文提出一种简单有效的自评估方法：将生成结果聚类为语义不同的组，转化为结构化多选题，利用模型对各选项的置信度作为不确定性度量。在多个模型和数据集上的实验表明，该方法持续优于基线，尤其在仅需2个额外样本时即表现优异，兼具高效性与有效性。
原文摘要 · Abstract (English)
Large language models (LLMs) demonstrate remarkable performance across diverse tasks, but they often generate responses that appear plausible while being factually incorrect. This problem is compounded by the lack of explicit uncertainty estimates, which makes it difficult for users to judge the reliability of model outputs. Existing uncertainty quantification methods typically rely on indirect signals, such as entropy across sampled generations. These signals can be difficult to interpret and do not fully leverage the model's ability to assess its own uncertainty. We propose a simple yet effective self-assessment method for uncertainty quantification in LLMs. Our approach groups sampled generations into semantically distinct clusters, converts them into answer options in a structured multiple-choice question, and uses the probability assigned by the LLM to each option as a confidence estimate. Experiments across multiple models and datasets show that our method consistently outperforms baseline approaches. Notably, it achieves competitive performance with as few as two additional samples, demonstrating both its effectiveness and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。