让拒绝的生成结果也参与后续推理,提升大模型生成效率
DFlow: Enabling Verifier Information Flow in Block Diffusion Speculative Decoding

- 用验证器隐藏状态引导后续生成,避免重复计算
- 在多个基准上使接受长度和生成质量优于DFlash
- 适合追求推理加速的LLM应用开发者
块扩散推测解码通过并行提出未来若干个词块,并用目标模型单次前向传播进行验证,提升了大语言模型推理效率。然而,现有方法仅保留被接受的前缀,丢弃被拒绝的后缀,导致这些位置的计算无法用于后续生成轮次,迫使生成器需从头重建未来词块表示。我们观察到,拒绝仅决定是否提交某个词,而验证器在被拒绝位置产生的隐藏状态仍可为后续预测提供有用信息。基于此,我们提出DFlow框架,实现验证器信息在生成轮次间的流动。DFlow复用目标验证器对被拒绝后缀产生的隐藏状态,指导后续生成,无需额外目标模型计算。为有效学习跨轮次的信息流动,引入自条件训练策略,将先前预测中的验证器表示反馈至后续预测。在Qwen3系列模型上多任务测试表明,DFlow在生成质量与接受长度上均持续优于DFlash。
原文摘要 · Abstract (English)
Block diffusion speculative decoding improves LLM inference efficiency by proposing a block of future tokens in parallel and verifying them with a single forward pass through the target model. However, existing methods retain only the accepted prefix and discard the rejected suffix, preventing the computation spent on these positions from benefiting subsequent drafting rounds and forcing the drafter to repeatedly reconstruct representations for future tokens from scratch. We observe that rejection only determines whether a proposed token can be committed, while the verifier representations at rejected positions can still provide useful information for subsequent predictions. Based on this observation, we propose DFlow, a simple yet effective framework that enables verifier information to flow across drafting rounds. DFlow reuses the hidden states produced by the target verifier for the rejected suffix to guide subsequent drafting without additional target computation. To effectively learn this information flow across drafting rounds, we introduce a self-condition train strategy that feeds verifier representations from earlier predictions back into subsequent predictions. Experiments on Qwen3 models across diverse benchmarks demonstrate that DFlow consistently improves draft quality and acceptance length over DFlash.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。