用嵌入空间的辐射分散度衡量大模型不确定性,简单有效且无需训练。
Distance Is All You Need: Radial Dispersion for Uncertainty Estimation in Large Language Models
- 基于生成文本在嵌入空间的分布离散程度,计算其与中心点的总距离。
- 在四个数据集上超越9个先进方法,对幻觉检测准确率提升显著。
- 无需模型内部信息,适合实际部署,尤其适合关注可信生成的开发者。
大语言模型(LLM)的不确定性检测对构建可靠系统至关重要,但现有方法往往复杂且依赖脆弱的语义聚类或模型内部访问。本文提出径向分散度评分(RDS),一种无需训练、完全模型无关的不确定性度量方法,通过计算在单位超球面上采样的N个生成文本嵌入的总L1距离与经验质心(均值嵌入)之间的距离,直接反映语义变异性。引入一种轻量级加权变体,利用模型自身词元概率,在样本生成时性能更优,超越九种最新基线方法。RDS还能自然扩展为每样本的不确定性估计,补充概率和一致性方法,同时保持轻量级以适应实际应用。在四个挑战性的自由问答数据集和四种LLM上,该方法实现了最先进的幻觉检测性能,且对样本数量和嵌入选择具有鲁棒性和可扩展性。代码已公开于 https://github.com/manhitv/RDS。
原文摘要 · Abstract (English)
Detecting uncertainty in large language models (LLMs) is essential for building reliable systems, yet many existing approaches are overly complex and depend on brittle semantic clustering or access to model internals. We introduce Radial Dispersion Score (RDS), a simple, training-free, fully model-agnostic uncertainty metric that measures the radial dispersion of sampled generations in embedding space. Specifically, given $N$ sampled generations embedded on the unit hypersphere, RDS computes the total l1 distance from the empirical centroid, i.e., the mean embedding, providing a direct geometric signal of semantic variability. A lightweight probability-weighted variant further incorporates the model's own token probabilities when available, outperforming nine recent state-of-the-art baselines. Moreover, RDS naturally extends to effective per-sample uncertainty estimates that complement probability- and consistency-based methods while remaining lightweight for practical use. Across four challenging free-form question-answering datasets and four LLMs, our metrics achieve state-of-the-art hallucination detection performance, while remaining robust and scalable with respect to sample size and embedding choice. These results highlight the practical value of RDS and its contribution toward improving the trustworthiness of LLMs. Code is publicly available at https://github.com/manhitv/RDS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。