用置信度控制推理步骤，让小模型快速验证，大模型只处理不确定项，提速近2.3倍且不丢精度。
ConfSpec: Efficient Step-Level Speculative Reasoning via Confidence-Gated Verification
- 小模型先猜推理步骤，高置信度直接通过，低置信度才交给大模型确认。
- 在多种任务上实现最高2.24倍加速，准确率与大模型一致。
- 无需额外裁判模型，可与词粒度推测解码叠加使用。
链式思维推理显著提升大语言模型在复杂任务上的表现，但因生成轨迹过长导致推理延迟高。步骤级推测推理旨在缓解该问题，但现有方法在准确性、推理速度和资源效率间存在长期权衡。我们提出 ConfSpec，一种基于置信度的级联验证框架，解决了这一权衡。核心洞察是生成与验证之间的不对称性：生成正确推理步骤需强大模型能力，而步骤级验证是受限的判别任务，小型草稿模型在其能力范围内具备良好校准性，能对高置信度决策直接接受，仅将不确定案例升至大型目标模型处理。跨多种工作负载的评估显示，ConfSpec 实现最高达 2.24× 的端到端加速，同时保持目标模型的准确率。该方法无需外部裁判模型，且与词粒度推测解码正交，可实现进一步乘法加速。
原文摘要 · Abstract (English)
Chain-of-Thought reasoning significantly improves the performance of large language models on complex tasks, but incurs high inference latency due to long generation traces. Step-level speculative reasoning aims to mitigate this cost, yet existing approaches face a long-standing trade-off among accuracy, inference speed, and resource efficiency. We propose ConfSpec, a confidence-gated cascaded verification framework that resolves this trade-off. Our key insight is an asymmetry between generation and verification: while generating a correct reasoning step requires substantial model capacity, step-level verification is a constrained discriminative task for which small draft models are well-calibrated within their competence range, enabling high-confidence draft decisions to be accepted directly while selectively escalating uncertain cases to the large target model. Evaluation across diverse workloads shows that ConfSpec achieves up to 2.24$\times$ end-to-end speedups while matching target-model accuracy. Our method requires no external judge models and is orthogonal to token-level speculative decoding, enabling further multiplicative acceleration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。