arXiv:2601.19278cs.CL2026-01被引 18

DART用并行生成加速大模型推理,比现有方法快30%以上。

DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference

  • 通过单次前向传播并行预测多个位置的词元概率,避免自回归推演
  • 在多个数据集上实现2.03倍至3.44倍的时钟时间提速
  • 适合追求高推理速度的部署场景,尤其适用于资源受限环境

推测解码是加速大语言模型推理的有效且无损方法。然而,现有的基于模型的草稿设计(如EAGLE3)为提升准确性需进行多步自回归推演,导致草稿阶段延迟过高,成为性能瓶颈。受扩散型大语言模型(dLLMs)启发,我们提出DART,利用并行生成降低草稿延迟。DART基于目标模型的隐藏状态,在单次前向传播中并行预测多个未来掩码位置的词元概率,从而消除草稿模型中的自回归滚动,同时保持轻量设计。基于这些并行概率预测,我们进一步引入一种高效的树剪枝算法,构建具有N-gram语义连续性的高质量草稿词元树。DART显著降低草稿阶段开销,同时保持高草稿准确率,带来显著提升的端到端解码速度。实验结果表明,DART在多个数据集上实现2.03倍至3.44倍的时钟时间加速,平均比EAGLE3快30%,提供了一种实用的推测解码框架。代码已开源于https://github.com/fvliang/DART。

原文摘要 · Abstract (English)

Speculative decoding is an effective and lossless approach for accelerating LLM inference. However, existing widely adopted model-based draft designs, such as EAGLE3, improve accuracy at the cost of multi-step autoregressive inference, resulting in high drafting latency and ultimately rendering the drafting stage itself a performance bottleneck. Inspired by diffusion-based large language models (dLLMs), we propose DART, which leverages parallel generation to reduce drafting latency. DART predicts logits for multiple future masked positions in parallel within a single forward pass based on hidden states of the target model, thereby eliminating autoregressive rollouts in the draft model while preserving a lightweight design. Based on these parallel logit predictions, we further introduce an efficient tree pruning algorithm that constructs high-quality draft token trees with N-gram-enforced semantic continuity. DART substantially reduces draft-stage overhead while preserving high draft accuracy, leading to significantly improved end-to-end decoding speed. Experimental results demonstrate that DART achieves a 2.03x--3.44x wall-clock time speedup across multiple datasets, surpassing EAGLE3 by 30% on average and offering a practical speculative decoding framework. Code is released at https://github.com/fvliang/DART.

大模型推理推测解码并行生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。