arXiv:2508.15842cs.CLcs.LG2025-08被引 10

通过分析推理链中的词汇线索,预测大模型回答错误的概率。

Lexical Hints of Accuracy in LLM Reasoning Chains

  • 用推理链中的词汇(如'guess'、'stuck')判断模型是否不确定。
  • 低准确率任务中,词汇不确定性信号比长度或情绪波动更有效。
  • 适合关注模型可靠性与安全部署的研究者和工程师。

在代码、数学和常识类基准上,通过强化学习微调大语言模型生成显式思维链(CoT)可显著提升性能。然而,在人类最后考试(HLE)这类当前准确率极低的任务上,模型常表现出高自信,反映其校准能力差。本文研究推理链的可度量特征是否能作为内部信心的可靠信号。分析三类特征:(i) 推理链长度,(ii) 链内情绪波动,(iii) 词汇线索(包括模棱两可词)。在 DeepSeek-R1 与 Claude 3.7 Sonnet 模型上测试 HLE(准确率≈9%)与 Omni-MATH(准确率≈70%)两个基准。结果表明,表达不确定性的词汇(如 'guess', 'stuck', 'hard')是预测错误最强信号;情绪变化提供较弱但互补的信号;推理链长度仅在中等难度的 Omni-MATH 上有信息量,在更难的 HLE 上无指示作用,说明其仅在模型能力范围内有效。此外,不确定性标记始终比高自信标记更显著,使错误比正确回答更容易预测。研究支持一种轻量级后处理校准机制,可补充不可靠的自报告概率,促进大模型的安全部署。

原文摘要 · Abstract (English)

Fine-tuning Large Language Models (LLMs) with reinforcement learning to produce an explicit Chain-of-Thought (CoT) before answering produces models that consistently raise overall performance on code, math, and general-knowledge benchmarks. However, on benchmarks where LLMs currently achieve low accuracy, such as Humanity's Last Exam (HLE), they often report high self-confidence, reflecting poor calibration. Here, we test whether measurable properties of the CoT provide reliable signals of an LLM's internal confidence in its answers. We analyze three feature classes: (i) CoT length, (ii) intra-CoT sentiment volatility, and (iii) lexicographic hints, including hedging words. Using DeepSeek-R1 and Claude 3.7 Sonnet on both Humanity's Last Exam (HLE), a frontier benchmark with very low accuracy, and Omni-MATH, a saturated benchmark of moderate difficulty, we find that lexical markers of uncertainty (e.g., $\textit{guess}$, $\textit{stuck}$, $\textit{hard}$) in the CoT are the strongest indicators of an incorrect response, while shifts in the CoT sentiment provide a weaker but complementary signal. CoT length is informative only on Omni-MATH, where accuracy is already high ($\approx 70\%$), and carries no signal on the harder HLE ($\approx 9\%$), indicating that CoT length predicts correctness only in the intermediate-difficulty benchmarks, i.e., inside the model's demonstrated capability, but still below saturation. Finally, we find that uncertainty indicators in the CoT are consistently more salient than high-confidence markers, making errors easier to predict than correct responses. Our findings support a lightweight post-hoc calibration signal that complements unreliable self-reported probabilities and supports safer deployment of LLMs.

大模型推理链校准错误预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。