推理模型通过逐步思考更准确表达信心,提升可信度。
Reasoning Models Better Express Their Confidence
- 让模型展开链式思考,逐步调整信心水平。
- 36组对比中33组信心校准优于非推理模型。
- 慢思考行为是信心准确的关键,适合高可靠场景使用。
尽管大型语言模型(LLMs)具有强大能力,但其信心表达常不准确,难以判断何时出错,限制了可靠性。本文发现,进行扩展链式思考(CoT)的推理模型不仅在解题上表现更优,还能更准确地表达信心。我们在六个数据集上对六种推理模型进行基准测试,发现在36个设置中的33个里,其信心校准显著优于非推理模型。详细分析表明,这种校准优势源于推理模型的慢思考行为(如探索多种解法、回溯修正),使其能在整个CoT过程中动态调整信心,逐步提高准确性。特别地,推理模型的信心校准随CoT推进而持续改善,而这一趋势在非推理模型中未出现。若移除慢思考行为,校准性能显著下降。此外,即使非推理模型仅通过上下文学习被引导慢思考,其校准能力也明显增强,充分证明慢思考是校准提升的核心原因。
原文摘要 · Abstract (English)
Despite their strengths, large language models (LLMs) often fail to communicate their confidence accurately, making it difficult to assess when they might be wrong and limiting their reliability. In this work, we demonstrate that reasoning models that engage in extended chain-of-thought (CoT) reasoning exhibit superior performance not only in problem-solving but also in accurately expressing their confidence. Specifically, we benchmark six reasoning models across six datasets and find that they achieve strictly better confidence calibration than their non-reasoning counterparts in 33 out of the 36 settings. Our detailed analysis reveals that these gains in calibration stem from the slow thinking behaviors of reasoning models (e.g., exploring alternative approaches and backtracking) which enable them to adjust their confidence dynamically throughout their CoT, making it progressively more accurate. In particular, we find that reasoning models become increasingly better calibrated as their CoT unfolds, a trend not observed in non-reasoning models. Moreover, removing slow thinking behaviors from the CoT leads to a significant drop in calibration. Lastly, we show that non-reasoning models also demonstrate enhanced calibration when simply guided to slow think via in-context learning, fully isolating slow thinking as the source of the calibration gains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。