arXiv:2608.14787cs.CRcs.CL2026-08

提出验证器跳过机制,用置信度调度提前提交预测文本,提升生成效率。

From Positionwise Confidence to Prefix Scheduling: Verifier Skipping in Speculative Decoding

  • 通过置信度信号决定是否跳过验证器,直接提交部分预测内容
  • 三种置信度方法均减少9.6%至13.5%的验证调用,保持相同通过率
  • 意外发现原始置信度表现最好,说明调度策略比预测精度更重要

推测解码通过小模型预提案头并由大模型并行验证来降低自回归生成成本。推测扩散解码(SDD)进一步通过离散扩散模型并行生成每个块内所有位置的文本,但仍在每一块上调用目标模型验证,造成瓶颈。本文提出验证器跳过策略:根据置信度选择性地直接提交部分草案前缀。研究发现,更好的词预测器未必带来更好调度器——跳过需连续高置信度前缀,短跳过反而增加额外生成轮次。在相同策略下,对比原始置信度、学习的边际和条件生存分数,三者在HumanEval上与DiffuCoder-7B-Instruct及Qwen3-32B配合时,均能节省9.6%至13.5%的验证调用,且保持相同的pass@1性能。令人意外的是,原始置信度表现最优;尽管边际生存分数在多数位置有更高位置级AUROC,但未在线上占优。分析表明,验证器跳过是有效的新型损失轴,其核心挑战在于前缀调度而非单纯词预测。

原文摘要 · Abstract (English)

Speculative decoding is a leading technique to reduce the cost of autoregressive generation by using a small drafter to propose several tokens, which are then verified in parallel by a larger target model. Speculative diffusion decoding (SDD) further removes sequential drafting by generating every position in a draft block in parallel with a discrete diffusion model. However, SDD still invokes the target on every block, leaving verification as a potential bottleneck. This paper recognizes that this creates a new control handle: whether to invoke the verifier at all. Thus, we study verifier skipping, a lossy policy that commits a selected draft prefix directly, and ask which confidence signal should schedule it. Interestingly, our study finds that better token predictors need not yield better schedulers: skips require contiguous high-confidence prefixes, while short skips can induce additional drafting rounds. To study this mismatch, we compare raw confidence with learned marginal and conditional survival scores under the same policy, using Strict SDD, lenience, and top-$k$ acceptance as baselines. On HumanEval with DiffuCoder-7B-Instruct and Qwen3-32B, all three confidence signals save $9.6\%$ to $13.5\%$ of verifier calls at the same observed pass@1 as Strict SDD. Surprisingly, raw confidence saves the most; marginal survival has higher positionwise AUROC than raw confidence at most positions, yet neither learned signal dominates online. Our analysis shows that verifier skipping is a useful new lossy axis and, surprisingly, its key challenge is prefix scheduling rather than token prediction alone.

推理优化生成调度扩散模型高效生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。