通过分层融合提升扩散式推测解码的生成能力,显著加速大模型推理。
DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding

- 每层草稿模型独立融合多层目标模型特征,增强表达能力。
- 在多个基准上实现最高5.52倍的推理加速,优于现有方法11%以上。
- 适合需要高效推理的大模型应用,如代码生成与数学推理。
块扩散推测解码通过同时预测整个块的所有词元来加速大语言模型推理,需具备足够强的草稿模型并有效利用目标模型内部知识。然而,当前最先进的DFlash方法将所有草稿层限制为共享单一融合表示,该表示仅来自少数目标层,抑制了每层的表达能力,阻碍草稿容量进一步扩展。本文提出 exttt{DFlare},通过轻量级分层融合机制打破这一瓶颈:每个草稿层可独立关注一组广泛的目标层的可学习组合,在几乎无额外开销下注入更丰富的目标知识,并为每层提供独特输入。这种增强的每层表达能力支持草稿模型向更深架构扩展,并保持稳定增益。我们还将训练数据从80万增至240万样本,以充分发挥扩大后的容量。在涵盖数学推理、代码生成和对话的六个基准上, exttt{DFlare}在Qwen3-4B、Qwen3-8B和GPT-OSS-20B上分别实现平均5.52倍、5.46倍和3.91倍的墙钟速度提升,较DFlash分别提升约11%、8%和5%。代码已开源于https://github.com/Tencent/AngelSlim。
原文摘要 · Abstract (English)
Block diffusion speculative decoding accelerates LLM inference by predicting all tokens within a block simultaneously for the target model to verify in parallel. Predicting an entire block at once requires a sufficiently capable draft model and effective utilization of the target model's internal knowledge. However, the state-of-the-art method DFlash constrains all draft layers to share a single fused representation derived from only a few target layers, limiting per-layer expressiveness and hindering further scaling of draft capacity. In this paper, we present \modelname, which flares out the narrow conditioning bottleneck of DFlash through a lightweight layer-wise fusion mechanism: each draft layer attends to its own learnable combination of a broad set of target layers at negligible overhead, simultaneously injecting richer target knowledge and providing every draft layer with a distinct input. This enhanced per-layer expressiveness enables scaling the draft model to deeper architectures with consistent gains. We further scale training data from 800K to 2.4M samples to fully exploit the enlarged capacity. On six benchmarks spanning mathematical reasoning, code generation, and conversation, \modelname attains average wall-clock speedups of 5.52x on Qwen3-4B, 5.46x on Qwen3-8B, and 3.91x on GPT-OSS-20B, improving over DFlash by roughly 11\%, 8\%, and 5\% respectively. Our code is available at https://github.com/Tencent/AngelSlim.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。