无需训练即可加速扩散语言模型,最高提速105倍。
DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding

- 通过动态连续验证消除概率依赖误差,实现安全加速
- 利用草稿引导跳跃解码,上下文扩展效率提升53.19倍
- 适合追求高效推理的开发者与研究者使用
尽管并行解码是扩散大语言模型(dLLMs)效率的核心,但当前策略常受过于保守的置信度阈值制约。这些阈值由联合概率依赖误差(JPDE)引发,导致冗余去噪迭代和推理速度不佳。为此,我们提出DC-Leap,一种无需训练的框架,可在中等置信度区间可靠加速dLLMs。DC-Leap引入动态连续验证机制,将严格有序的因果约束融入并行解码过程,通过逐步验证标记依赖关系有效消除JPDE,实现可靠加速且性能相当。此外,该框架结合草稿引导解码,利用草稿跳过多个标记向前推进上下文,提供前瞻信息,并在推理中保留双向注意力的结构优势。在标准基准上的大量实验表明,DC-Leap在长序列生成任务上实现显著提速:在MBPP上最高达53.19倍;当结合KV-Cache时,最高速度提升可达105.02倍,且生成质量相当。代码已公开于https://github.com/ffh-wyls/DC-Leap。
原文摘要 · Abstract (English)
While parallel decoding is central to the efficiency of Diffusion Large Language Models (dLLMs), current strategies are often hindered by overly conservative confidence thresholds. These thresholds, necessitated by the Joint Probability Dependence Error (JPDE), result in redundant denoising iterations and suboptimal inference speeds. To overcome this, we propose DC-Leap, a training-free framework that enables reliable acceleration of dLLMs in the moderate-confidence regime. DC-Leap introduces a Dynamic Contiguous Verification strategy that integrates strictly-ordered causal constraints into the parallel decoding process. By progressively validating token dependencies, this mechanism effectively neutralizes the JPDE, enabling reliable acceleration with comparable performance. Furthermore, DC-Leap incorporates the draft-guided decoding mechanism, where the draft helps extend the context by leaping forward across multiple tokens, providing look-ahead context and retaining the structural benefits of bidirectional attention during inference. Extensive experiments on standard benchmarks demonstrate that DC-Leap achieves substantial speedups, up to 53.19x on MBPP for long-sequence generation, and up to 105.02x when combined with KV-Cache with comparable generation quality. Code is available at https://github.com/ffh-wyls/DC-Leap .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。