用结构化图谱加速扩散型大模型推理，提升生成速度且不改变输出结果。
Structuring The Future: Diffusion LLM Speculative Decoding via Calibrated Draft Graphs
- 构建双向块状生成的草稿图，实现自动推测并减少独立草稿模型开销。
- 在多个模型上实现最高8.6倍推理次数减少、6.3倍令牌生成加速。
- 适合追求高吞吐量且需保持输出一致性的扩散模型部署场景。
扩散型大语言模型（dLLMs）作为自回归模型的有力替代，具备显著更高的令牌生成速率潜力。为释放这一潜力，本文提出Spiffy——一种推测解码算法，在保证模型输出分布不变的前提下加速dLLM推理。该工作解决了将自回归模型的推测解码思想应用于dLLMs所面临的独特挑战。Spiffy通过自动推测生成结构化的草稿状态，以新颖的有向草稿图形式利用dLLM生成的双向、分块特性。草稿图在离线阶段进行校准以最大化接受率，并在推理过程中动态剪枝以提升计算效率。我们详细阐述了Spiffy的数学形式，并在结合键值缓存与阈值动态去掩码策略下，验证其对LLaDA、Dream和SDAR模型的加速效果，实现最多8.6倍的模型推理次数减少和6.3倍的令牌生成速率提升。
原文摘要 · Abstract (English)
Diffusion LLMs (dLLMs) have recently emerged as a powerful alternative to autoregressive LLMs (AR-LLMs) with the potential to operate at significantly higher token-generation rates. To unlock this potential, we present Spiffy, a speculative decoding algorithm to accelerate dLLM inference while provably preserving the model's output distribution. This work addresses the unique challenges involved in applying ideas from speculative decoding of AR-LLMs to dLLMs. Spiffy performs auto-speculation to eliminate the overheads of an independent draft model, structuring draft states in the form of a novel directed draft graph to take advantage of the bidirectional, blockwise nature of dLLM generation. These draft graphs are calibrated offline to maximize acceptance rates and are dynamically pruned during inference for improved computational efficiency. We present a detailed formulation of Spiffy and demonstrate its ability to accelerate LLaDA, Dream, and SDAR models in combination with KV caching and threshold-based dynamic unmasking leading to up to $8.6\times$ reduction in model inferences and $6.3\times$ acceleration in token rate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。