arXiv:2608.30427cs.CLcs.LG2026-08

通过改进草稿模型,提升生成速度并发现隐藏的加速潜力。

Ceiling-Clipped Acceptance Histograms Indicate Stranded Speed-up in Block-Diffusion Speculative Decoding

  • 用接受率直方图识别草稿模型未充分利用的加速空间。
  • 提出DBloom方法,使块大小从16扩至24后,每提示平均多生成0.8~1.37个词。
  • 适用于追求高效生成的模型部署者,尤其在高吞吐场景下有显著收益。

推测解码通过一个高效的草稿模型(drafter)在单次传递中提出候选词供目标模型验证,保持目标模型输出分布不变。像DFlash和DFlare这样的高接受率块扩散草稿模型可在一次并行传递中填满整个块。在许多循环中,目标模型会接受整个块,导致草稿模型在其训练的块范围耗尽前验证失败。我们称这种未实现的接受为“搁置加速”。平均提交长度掩盖了这一现象,而接受率直方图则通过在顶峰箱(即完全接受块的循环比例)中出现尖峰暴露了它。我们建议将直方图作为训练前检查工具。盲目扩大推理时的块大小无法恢复加速,因为一旦块超过训练尺寸,草稿模型的双向注意力会改变早期位置的分布,削弱块前端验证效果。为此,我们提出在更长块上对草稿模型进行短课程微调,称为DBloom。将预训练的DFlash和DFlare草稿模型从块大小16扩展到24,在Qwen3-8B和Qwen3-4B目标模型上,高顶峰基准下的每提示提交长度中位数提高0.8个词(最多+1.1)。若先进行连续性微调再扩展,增幅达1.37词。相同扩展在Gemma-4-12B-IT的七个基准上,中位数提升0.41词(臂A),完整流程(臂B)额外增加0.29至0.98词。与同代基于树结构的JetSpec对比,在提示匹配条件下,所有基准上在树预算不超过64节点时,DBloom均提交更多词。

原文摘要 · Abstract (English)

Speculative decoding speeds up generation with an efficient draft model (drafter) that proposes tokens for a target model to verify in one pass, preserving the target's output distribution. High-acceptance block-diffusion drafters such as DFlash and DFlare fill an entire block in one parallel pass. In many cycles, the target accepts the whole block, so the drafter exhausts its trained block horizon before verification fails. We call this unrealized acceptance stranded speed-up. A mean committed length, per prompt or per cycle, hides it, whereas the acceptance histogram exposes it as a spike in the ceiling bin, the fraction of cycles that accept the entire block. We recommend the histogram as a preflight check before spending training compute. Naively widening the block at inference does not recover the speed-up, because once the block outgrows its training size, the drafter's bidirectional attention shifts its distribution even at early positions and erodes front-of-block verification. Instead, we post-train the drafter on a longer block with a short curriculum that emphasizes the newly exposed positions, a method we call DBloom. Expanding the pretrained DFlash and DFlare drafters from block size 16 to 24 across Qwen3-8B and Qwen3-4B targets raises the per-prompt committed length on the high-ceiling benchmarks by a median of +0.8 tokens (up to +1.1). Once continuation fine-tuning precedes expansion, the increase reaches 1.37 tokens. The same expansion also lifts committed length on all seven benchmarks for Gemma-4-12B-IT, a different model family, by a median of +0.41 tokens (Arm A), and the full continuation-then-expand pipeline (Arm B) adds +0.29 to +0.98 tokens over the same B16 drafter. In a prompt-matched comparison against JetSpec, a contemporary tree-based drafter not used in our design, DBloom commits more tokens on every benchmark at tree budgets up to 64 nodes.

生成加速扩散模型推测解码模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。