用语义聚类法高效评估大模型输出的可信度。
Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Models
- 通过嵌入聚类和前缀匹配将词元分组到语义一致簇中
- 单次生成即可量化不确定性，性能接近顶尖方法
- 适合需要快速可靠判断的部署场景
大语言模型在各类任务中表现出色，但其输出的真实性无法保证，且易过度自信，影响可靠性。不确定性量化为识别不可靠输出提供了有效途径，但现有方法多依赖重复采样或额外模型，带来显著计算开销。为此，我们提出语义词元聚类（STC），利用大模型内在的语义信息实现高效不确定性量化。具体地，通过嵌入聚类与前缀匹配将词元划分为语义一致的簇，并基于对应语义簇内的概率质量聚合来衡量不确定性。该方法仅需一次生成，无需辅助模型。实验表明，STC在保持与最先进方法相当性能的同时，大幅降低计算开销。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated remarkable capabilities across diverse tasks. However, the truthfulness of their outputs is not guaranteed, and their tendency toward overconfidence further limits reliability. Uncertainty quantification offers a promising way to identify potentially unreliable outputs, but most existing methods rely on repeated sampling or auxiliary models, introducing substantial computational overhead. To address these limitations, we propose Semantic Token Clustering (STC), an efficient uncertainty quantification method that leverages the semantic information inherently encoded in LLMs. Specifically, we group tokens into semantically consistent clusters using embedding clustering and prefix matching, and quantify uncertainty based on the probability mass aggregated over the corresponding semantic cluster. Our approach requires only a single generation and does not depend on auxiliary models. Experimental results show that STC achieves performance comparable to state-of-the-art baselines while substantially reducing computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。