调整大模型自信评分尺度，能显著提升其自我认知准确性。
Rescaling Confidence: What Scale Design Reveals About LLM Metacognition
- 通过系统实验调整评分粒度、边界和范围，研究尺度设计影响
- 0-20量表比常用0-100量表提升元认知敏感性，效果更优
- 模型偏好整数分值，即使在非规则量表下仍存在此倾向
口头化自信评分（即大模型报告数值确定性分数）被广泛用于黑箱场景下的不确定性估计，但常用的0-100评分尺度本身很少被深入考察。我们发现这一设计并非中立：在六种大模型与三个数据集上，超过78%的评分集中在三个整数点上，表现出严重离散化。为探究该现象，我们系统地操纵了三个维度的评分尺度：粒度、边界位置与范围规律性，并使用meta-d'评估元认知敏感性。结果表明，0-20量表在所有条件下均显著优于标准0-100格式；边界压缩会降低性能，且即便在不规则范围下，整数偏好依然存在。这说明评分尺度设计直接影响口头化不确定性的质量，应作为大模型评估中的关键变量加以考量。
原文摘要 · Abstract (English)
Verbalized confidence, in which LLMs report a numerical certainty score, is widely used to estimate uncertainty in black-box settings, yet the confidence scale itself (typically 0--100) is rarely examined. We show that this design choice is not neutral. Across six LLMs and three datasets, verbalized confidence is heavily discretized, with more than 78\% of responses concentrating on just three round-number values. To investigate this phenomenon, we systematically manipulate confidence scales along three dimensions: granularity, boundary placement, and range regularity, and evaluate metacognitive sensitivity using $meta\text{-}d'$. We find that a 0--20 scale consistently improves metacognitive efficiency over the standard 0--100 format, while boundary compression degrades performance and round-number preferences persist even under irregular ranges. These results demonstrate that confidence scale design directly affects the quality of verbalized uncertainty and should be treated as a first-class experimental variable in LLM evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。