arXiv:2608.29748cs.CL2026-08

让被拒的生成轨迹继续参与后续推理,提升大模型解码速度。

ReTrace: Rejected-Trajectory Conditioning for Speculative Decoding

  • 用前一轮被拒的生成内容作为新生成块的条件,避免浪费计算
  • 在不改变验证机制的前提下,平均接受长度提升23.7%以上
  • 适合追求高效推理的模型部署者,尤其适用于长文本生成场景

推测解码通过轻量级草稿模型生成多个候选词,由大型目标模型并行验证以加速自回归语言模型推理。然而,在首次拒绝后,标准前缀验证会丢弃剩余草稿后缀,导致已生成和验证位置的计算无法贡献于解码进度。针对DFlash,我们发现被拒绝后缀中的位置仍可能与目标延续对齐,表明草稿模型即使存在局部词级别错误,仍保留有用语义与结构信息。受条件扩散启发,我们提出ReTrace:将每个草稿块的生成条件从全新掩码占位符改为依赖前一轮被拒后缀。ReTrace保留被拒后缀的隐藏表示,将其与下一草稿块对齐,利用同一验证轮次的目标感知修正信号进行精炼,并通过门控残差融合将其引入草稿器输入嵌入。由于被拒标记永不提交,且目标侧验证不变,ReTrace保持推测解码无损性,无需额外模型前向传播。在Qwen3模型上,涵盖数学推理、代码生成和开放式对话的任务中,实验表明ReTrace持续提升了平均接受长度和端到端解码速度,优于其基础模型DFlash。通过引入跨轮次条件化,而不修改轮内提案生成,ReTrace与现有草稿优化方法基本正交,可进一步结合使用以获得更大收益。

原文摘要 · Abstract (English)

Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which are then verified in parallel by a larger target model. However, after the first rejection, standard prefix-based verification discards the remaining draft suffix, so the computation spent generating and verifying those positions does not contribute to decoding progress. Focusing on DFlash, we show that rejected positions in a rejected suffix may still align with the target continuation, indicating that the draft model can retain useful semantic and structural information despite local token-level errors. Motivated by this observation and inspired by conditional diffusion, we introduce~\textbf{ReTrace}, a rejected-trajectory conditioning method that conditions each draft block on the rejected suffix from the previous round rather than generating it from fresh mask placeholders alone. ReTrace retains the hidden representations of the rejected suffixes, aligns them with the next draft block, refines them using target-aware correction signals from the same verification pass, and admits them into the drafter's input embeddings through gated residual fusion. Because rejected tokens are never committed and target-side verification remains unchanged, ReTrace preserves the lossless property of speculative decoding without requiring an additional model forward pass. Experiments with Qwen3 models across mathematical reasoning, code generation, and open-ended dialogue demonstrate that ReTrace consistently improves average acceptance length and end-to-end decoding speed over its DFlash backbone. By introducing cross-round conditioning without modifying within-round proposal generation, ReTrace is largely orthogonal to existing drafting improvements and might be combined with them for further gains.

推理加速推测解码模型优化大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。