arXiv:2607.05721cs.CL2026-07

为大模型生成结果提供细粒度不确定性评估,精准定位错误所在。

SpanUQ: Span-Level Uncertainty Quantification for Large Language Model Generation

  • 通过检测语义连贯片段并估计其置信度,实现更精细的不确定性分析。
  • 在五个主流大模型上表现最优,速度比采样方法快10-20倍,F1达0.910。
  • 适用于需要可信生成与自我修正的场景,如医疗、法律等高风险领域。

不确定性估计对大语言模型(LLM)的可信部署及自修正生成至关重要。然而现有方法粒度不佳:词元级评分缺乏语义一致性,序列级评分无法精确定位错误。本文提出新的任务——段级不确定性估计(SLUE),聚焦于语义连贯文本片段这一自然粒度,每个片段表达单一可评估意义单元。为此,我们设计SPAUQ,一种轻量级探测器,从昂贵的多样本推理中蒸馏不确定性知识,仅需一次前向传播即可完成。SPANUQ采用DETR式片段解码器,通过混合贝塔分布同时检测片段并估计不确定性,训练结合贝塔负对数似然回归与对比排序目标。我们构建了SPANUQ-BENCH,首个段级不确定性基准,包含20,000个提示、293,000个标注片段和基于多样本断言验证生成的连续软标签。在五个不同骨干模型上的实验表明,SPANUQ在段级不确定性质量上持续领先,优于最强探测基线及所有采样方法,且速度快10–20倍。其基于DETR的片段检测器达到0.910 F1,比最佳启发式方法提升39.4%,实现序列级方法无法提供的精确错误定位。该框架在两个模型家族的五种大模型上具有良好的泛化能力。

原文摘要 · Abstract (English)

Uncertainty estimation is essential not only for the trustworthy deployment of large language models (LLMs) but also as a foundation for self-refinement in LLM generation. However, existing approaches operate at suboptimal granularities: token-level scores lack semantic coherence, while sequence-level scores fail to localize errors. We formalize Span-Level Uncertainty Estimation (SLUE), a new task that targets the natural granularity for uncertainty: semantically coherent text spans, each conveying a single assessable unit of meaning. To address this task, we introduce SPANUQ, a lightweight probe that distills the uncertainty knowledge from expensive multi-sample inference into a single forward pass over LLM hidden states. SPANUQ employs a DETR-style span decoder to simultaneously detect spans and estimate their uncertainty via a Mixture of Beta distribution, trained with a principled combination of Beta NLL regression and contrastive ranking objectives. We construct SPANUQ-BENCH, the first span-level uncertainty benchmark comprising 20K prompts, 293K annotated spans, and continuous soft labels derived from multi-sample claim verification. Experiments on five LLM backbones show that SPANUQ consistently achieves the best span-level uncertainty quality, outperforming the strongest probe baseline and all sampling-based methods while being 10-20x faster. Its DETR-based span detector attains 0.910 F1, surpassing the best heuristic by 39.4%, enabling precise error localization that sequence-level methods cannot provide. The framework generalizes across five LLMs spanning two model families.

不确定性量化大模型生成误差定位

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。