通过最大化模型自我确定性,用少量采样提升推理准确率。
Improving reasoning at inference time via uncertainty minimisation
- 在每步推理中选最自信的思考路径,基于模型内部预测分布。
- 小样本下性能超越贪心解码,媲美或超过自一致性方法。
- 适合追求高效推理的开发者,尤其跨语言场景表现稳健。
大型语言模型(LLMs)已具备强大的多步推理能力,但现有推理时扩展方法计算成本高,常依赖大量采样或外部评估器。本文提出一种以不确定性最小化为核心的原理性策略,作用于个体思维而非词元层面。该方法在每一步推理中选择使模型自确定性最大化的延续路径,自确定性由模型内部预测分布计算得出。此方法仅需少量样本即可显著提升性能,完全依赖模型内信号,适用于开放式问题,不同于多数投票等方法。在 MATH500 与 GSM8K 多个模型规模上的实验表明,思维级自确定性最大化持续优于贪心解码,并在相近词元预算下匹配或超越自一致性方法。跨语言评估进一步显示,该方法在低资源语言上仍具鲁棒性。对自确定性动态的分析表明,正确推理路径会早期收敛至稳定轨迹,提示早期决策(可能关联推理规划)可预测最终准确性。基于此,我们发现仅对早期步骤应用自确定性最大化,即可解释大部分性能提升,提供一种简单高效的推理时扩展方法。
原文摘要 · Abstract (English)
Large language models (LLMs) now exhibit strong multi-step reasoning abilities, but existing inference-time scaling methods remain computationally expensive, often relying on extensive sampling or external evaluators. We propose a principled strategy that frames reasoning as uncertainty minimisation and operates at the level of individual thoughts rather than tokens. Our method selects, at each reasoning step, the continuation that maximizes the model's self-certainty, a metric computed from its internal predictive distribution. This approach achieves significant improvement with a small number of samples, relies exclusively on model-internal signals, and applies to open-ended questions as opposed to methods like majority voting. Experiments on MATH500 and GSM8K across multiple model sizes demonstrate that thought-level self-certainty maximization consistently outperforms greedy decoding and matches or exceeds self-consistency under comparable token budgets. Cross-linguistic evaluations further indicate that the method transfers robustly beyond high-resource languages. Furthermore, analysis of self-certainty dynamics reveals that correct reasoning trajectories converge early to stable paths, suggesting that early decisions, likely associated with the planning of the reasoning process, are predictive of final accuracy. Building on this result, we show that self-certainty maximisation applied to the early steps can explain most of the performance gain and provide a simple yet efficient inference-time scaling method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。