用信心控制自我修正,大幅减少推理时的计算量
CoRefine: Confidence-Guided Self-Refinement for Adaptive Test-Time Compute
- 基于置信度动态决定是否停止、重检或换方法
- 平均每题仅需2.7次修正,节省190倍计算资源
- 适合需要高效推理的智能体和验证能力有限的场景
大型语言模型常通过并行解码(如512个样本)提升推理准确率,但计算开销巨大。本文提出CoRefine,一种基于置信度的自修正方法,采用轻量级211k参数的Conv1D控制器,在冻结LLM基础上实现高精度推理。控制器利用完整置信度轨迹判断是否终止、重检或尝试新路径,平均每题仅需2.7次修正,相比512样本基线节省约190倍令牌。在多个推理基准和三款开源模型上,控制器在自信终止时达到92.6%的精确率,表明置信度变化可有效指示正确性,无需真实标签。进一步提出CoRefine-Tree,结合串行与并行策略,自适应平衡探索与利用,支持便捷部署与验证器兼容。通过将置信度作为控制信号而非正确性保证,CoRefine为可扩展推理和代理场景提供了模块化基础。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often rely on test-time scaling via parallel decoding (for example, 512 samples) to boost reasoning accuracy, but this incurs substantial compute. We introduce CoRefine, a confidence-guided self-refinement method that achieves competitive accuracy using a fraction of the tokens via a lightweight 211k-parameter Conv1D controller atop a frozen LLM. The controller consumes full-trace confidence to decide whether to halt, re-examine, or try a different approach, enabling targeted self-correction with an average of 2.7 refinement steps per problem and roughly 190-fold token reduction relative to 512-sample baselines. Across diverse reasoning benchmarks and three open-source models, the controller achieves 92.6 percent precision when it confidently halts, indicating that confidence dynamics reliably signal correctness without ground-truth verification. We extend this to CoRefine-Tree, a hybrid sequential-parallel variant that adaptively balances exploration and exploitation, with easy serving integration and verifier compatibility. By treating confidence as a control signal rather than a correctness guarantee, CoRefine provides a modular primitive for scalable reasoning and agentic settings with imperfect verifiers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。