arXiv:2603.17839cs.CLcs.AI2026-03被引 16

大模型的自信评分是自动计算并存储的,而非事后编造。

How do LLMs Compute Verbal Confidence

  • 信心值在生成答案时就已计算并缓存,后续才输出。
  • 缓存的信心信息比词汇概率更能解释评分差异。
  • 适合研究模型自我评估机制或提升可信度的读者。

言语自信——即要求大模型以数字或类别形式表达自身信心——被广泛用于提取黑箱模型的不确定性估计。然而,大模型如何内部生成此类评分仍不明确。本文聚焦Gemma 3 27B(TrivQA、BigMath、MMLU)、Qwen 2.5 7B及推理模型Magistral Small 24B,回答两个问题:信心是在被询问时即时计算,还是在生成答案时自动计算并缓存?以及信心代表什么——仅仅是词元对数概率,还是更复杂的答案质量评估?通过激活引导、修补、噪声注入和替换实验,我们发现信心表示在答案相邻位置先出现,再传递至输出位置。注意力屏蔽定位了信息流:信心从答案词元中汇聚,在首个答案后位置缓存,随后检索输出。线性探测与方差分解表明,这些缓存表示能解释超出词元概率的大量信心变异,说明其反映的是更丰富的答案质量评估,而非简单流畅性读取。结果表明,言语自信是自动、复杂的自我评估,而非事后重构,对理解大模型的元认知与改进校准具有重要意义。

原文摘要 · Abstract (English)

Verbal confidence -- prompting LLMs to state their confidence as a number or category -- is widely used to extract uncertainty estimates from black-box models. However, how LLMs internally generate such scores remains unknown. We address two questions: first, when confidence is computed -- just-in-time when requested, or automatically during answer generation and cached for later retrieval; and second, what verbal confidence represents -- token log-probabilities, or a richer evaluation of answer quality? Focusing on Gemma 3 27B (across TriviaQA, BigMath, and MMLU), Qwen 2.5 7B, and the reasoning model Magistral Small 24B, we provide convergent evidence for cached retrieval. Activation steering, patching, noising, and swap experiments reveal that confidence representations emerge at answer-adjacent positions before appearing at the verbalization site. Attention blocking pinpoints the information flow: confidence is gathered from answer tokens, cached at the first post-answer position, then retrieved for output. Critically, linear probing and variance partitioning reveal that these cached representations explain substantial variance in verbal confidence beyond token log-probabilities, suggesting a richer answer-quality evaluation rather than a simple fluency readout. These findings demonstrate that verbal confidence reflects automatic, sophisticated self-evaluation -- not post-hoc reconstruction -- with implications for understanding metacognition in LLMs and improving calibration.

大模型自信自省机制元认知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。