让大模型生成时实时评估信心,避免盲目自信。
Mind the Generation Process: Fine-Grained Confidence Estimation During LLM Generation
- 构建训练数据捕捉模型输出概率分布,监督学习信心分数。
- 推理时利用后续文本反向增强当前序列的信心估计。
- 可灵活选择信心评估位置,适合需要可信生成的场景。
尽管大语言模型在多种任务中表现卓越,但缺乏自我意识,常对错误预测赋予过高信心。准确的信心估计对提升生成结果的可信度至关重要。然而,现有方法多为粗粒度评分,无法在生成过程中提供连续、精细的信心值。为此,我们提出FineCE,一种在生成过程中实现精确细粒度信心估计的新方法。首先,设计完整的数据构建流程,有效捕捉模型响应的潜在概率分布;随后,训练一个模型以监督方式预测任意文本序列的信心分数。此外,提出反向信心融合(BCI)策略,在推理时利用后续文本信息增强当前序列的信心估计。我们还引入三种策略,用于确定生成过程中的最优信心评估位置。在多个基准数据集上的实验表明,FineCE始终优于现有经典方法。代码及所有基线均已开源。
原文摘要 · Abstract (English)
While large language models (LLMs) have demonstrated remarkable performance across diverse tasks, they fundamentally lack self-awareness and frequently exhibit overconfidence, assigning high confidence scores to incorrect predictions. Accurate confidence estimation is therefore critical for enhancing the trustworthiness and reliability of LLM-generated outputs. However, existing approaches suffer from coarse-grained scoring mechanisms that fail to provide fine-grained, continuous confidence estimates throughout the generation process. To address these limitations, we introduce FineCE, a novel confidence estimation method that delivers accurate, fine-grained confidence scores during text generation. Specifically, we first develop a comprehensive pipeline for constructing training data that effectively captures the underlying probabilistic distribution of LLM responses, and then train a model to predict confidence scores for arbitrary text sequences in a supervised manner. Furthermore, we propose a Backward Confidence Integration (BCI) strategy that leverages information from the subsequent text to enhance confidence estimation for the current sequence during inference. We also introduce three strategies for identifying optimal positions to perform confidence estimation within the generation process. Extensive experiments on multiple benchmark datasets demonstrate that FineCE consistently outperforms existing classical confidence estimation methods. Our code and all baselines used in the paper are available on GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。