用不确定性引导轻量级纠错循环，小模型逼近大模型效果
Entropy-Guided Loop: Achieving Reasoning through Uncertainty-Aware Generation
- 基于分词级不确定性触发单次精准修正
- 在31%响应中实现准确率提升16个百分点
- 适合对成本与精度平衡有要求的部署场景
推理模型虽性能更优，但代价高出3-5倍且延迟增加。本文提出熵引导的测试时循环：通过分词级不确定性触发一次针对性修正。提取概率分布，计算前k个候选的香农熵，并结合困惑度、最大熵和低置信度词数，采用简单或逻辑判断是否触发修正。不同于仅用熵进行度量或解码的方法，本方法将紧凑的不确定报告（包含词、置信度、候选及上下文）回传给模型以指导修正。在涵盖技术问答、数学与代码生成任务的多个基准上，加入该循环的小模型可达到参考推理模型95%的性能，成本仅为三分之一。该方法在约31%的输出中进行选择性修正，相比单次推理准确率提升16个百分点。结果表明，这种不确定性感知的循环在单次推理与高成本推理链之间提供了有效折中，适用于对质量与成本均敏感的生产环境。
原文摘要 · Abstract (English)
Reasoning models often outperform smaller models but at 3--5$\times$ higher cost and added latency. We present entropy-guided refinement: a lightweight, test-time loop that uses token-level uncertainty to trigger a single, targeted refinement pass. We extract logprobs, compute Shannon entropy on top-$k$ alternatives, and apply a simple OR-logic trigger over perplexity, maximum token entropy, and low-confidence-token count. Unlike approaches that use entropy only for measurement or decoding, we pass a compact uncertainty report (tokens, confidences, alternatives, context) back to the model to guide corrective edits. On representative technical queries across reasoning, mathematics, and code generation tasks, a small model with our loop approaches 95\% of a reference reasoning model's quality at approximately one-third of the cost. The method achieves selective refinement on ~31\% of responses while improving accuracy by 16 percentage points over single-pass inference. We demonstrate that this uncertainty-aware loop provides an effective middle ground between single-pass inference and expensive reasoning chains, making it practical for production deployments where both quality and cost matter.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。