arXiv:2607.19223cs.LGcs.CL2026-07

改进扩散采样生成,让大模型推理更快更稳。

AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters

论文配图:AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters
图 1 · 摘自论文原文
  • 用在线策略蒸馏算法降低不同领域间的生成波动。
  • 动态调整生成长度,减少验证开销,提升吞吐量66%。
  • 适合高并发场景,特别适用于追求稳定高速的部署需求。

推测解码通过轻量级草稿模型先生成候选序列,再由目标模型并行验证,已成为加速大语言模型推理的主流方法。近期如DFlash等方法利用扩散草稿模型,借助并行去噪机制实现单次前向传播生成,显著提升效率。然而本文发现,扩散草稿模型中的双向注意力虽支持全局上下文建模与并行生成,但其固有的全局依赖性导致域级与词元级的高方差:不同领域间接受率波动大,不同位置词元质量不均。为此,我们提出AdaFlash框架,包含两个核心组件:(i) 针对扩散草稿模型设计的基于反向KL散度的在线策略蒸馏(OPD)算法,实现稳定收敛,有效降低域级方差;(ii) 自适应长度头,动态调整候选序列长度,显著降低目标模型验证成本,缓解词元级方差。实验表明,AdaFlash在部署中持续提升加速比,尤其在高并发场景下表现优异,相比现有最先进方法,吞吐量最高提升约66%。

原文摘要 · Abstract (English)

Speculative decoding, in which a lightweight draft model first generates a draft sequence that is then verified in parallel by the target model, has become a prevalent paradigm for accelerating large language model inference. Recent work such as DFlash further boosts drafting efficiency by leveraging diffusion drafters, whose parallel denoising mechanism enables draft generation in a single forward pass. In this work, we uncover a central pitfall of diffusion drafters: bidirectional attention is a double-edged sword. On one hand, it endows the model with parallel generation and global contextual modeling capabilities; on the other hand, this inherent global dependency introduces high variance at both the domain-level and the token-level: acceptance rates fluctuate substantially across different domains, and draft token quality also varies heterogeneously at different token positions. To tackle this issue, we propose AdaFlash framework, comprising two components: (i) an on-policy distillation (OPD) algorithm with reverse-KL divergence tailored for diffusion drafters, bringing stable convergence and effectively reducing domain-level variance; and (ii) an adaptive length head that dynamically adjusts the candidate sequence length on the fly, substantially lowering the verification cost of the target model and handling token-level variance. Experiments demonstrate that AdaFlash consistently improves speedup rate during deployment, with especially significant gains in high-concurrency scenarios, achieving up to approximately 66% higher throughput than previous state-of-the-art methods.

推理加速扩散模型生成优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。