用熵值指导生成,让大模型在不确定处多思考,提升推理准确性。
Think Twice Before You Write -- an Entropy-based Decoding Strategy to Enhance LLM Reasoning

- 根据每个词的不确定性动态选择分支点,只在模糊处深入探索。
- 在GSM8K等数据集上表现优于传统方法,小模型接近GPT-5效果。
- 适合资源有限但需高可靠推理的应用场景,如边缘设备部署。
解码策略在塑造大语言模型推理能力中起关键作用。传统方法如贪心解码和束搜索易产生错误传播,采样方法缺乏鲁棒性。自一致性通过聚合多个推理路径提升可靠性,但计算开销大。本文提出一种基于熵的解码框架,在每一步计算词分布熵,识别高不确定性位置,仅在这些薄弱环节进行选择性分支。维护一个动态的部分推理路径池,持续扩展直至完成解答,使计算集中在最不确定区域,避免在确定区域无效探索。为实现高效终止,采用推理完成后评估的熵后停止准则(EAT),而非逐步增量判断。在GSM8K、AMC2023及其扰动变体上的实验表明,该方法始终表现优异。尤其在小规模模型上,性能可媲美GPT-5,而成本仅为几分之一。
原文摘要 · Abstract (English)
Decoding strategies play a central role in shaping the reasoning ability of large language models (LLMs). Traditional methods such as greedy decoding and beam search often suffer from error propagation, while sampling-based approaches introduce randomness without adequate robustness. Self-consistency improves reliability by aggregating multiple rollouts, but incurs significant computational overhead. We propose an entropy-guided decoding framework that introduces token-level adaptivity into generation. At each step, the model computes the entropy of the token distribution, identifies high-uncertainty positions, and selectively branches on these vulnerable points. A dynamic pool of partial rollouts is maintained and expanded until solutions are completed, concentrating computation where uncertainty is greatest and avoiding unnecessary exploration in confident regions. To enable efficient termination, we apply a rollout-level Entropy After </Think> (EAT) stopping criterion by performing entropy evaluation after the full reasoning trace, rather than incrementally at every step. Experiments on GSM8K, AMC2023, and their perturbed variants demonstrate that our method achieves consistently strong accuracy. Notably, on smaller LLMs, performance is comparable to GPT-5 while operating at a fraction of the cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。