通过并行推测解码，大幅提升扩散型大模型生成效率。
PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
- 基于置信度分数动态选择解码位置，无需额外模型调用
- 单次前向传播实现最多5.5倍的每步生成词数提升
- 适合追求高效推理的代码与逻辑推理场景
扩散型大语言模型（dLLMs）通过迭代去噪被掩码的词元序列生成文本。尽管每步可并行预测所有被掩码位置，但大量去噪步骤仍导致推理成本高昂。可通过每步解码多个词元来降低空间开销，或通过合并多个去噪步骤为一次验证调用以减少时间开销。本文提出无需训练的并行推测解码（PSD）框架，同时优化这两个维度。利用单次前向传播获得的置信度分数，PSD采用可配置的自适应解码策略选择解码位置，并构建多深度推测草稿，无需额外模型调用。最终通过批量分层验证，保留与更新预测一致的最深草稿。在三个dLLMs上进行的实验表明，PSD在推理效率与生成质量间取得良好权衡，最高实现每前向传播生成5.5倍的词元，且准确率接近贪婪解码。
原文摘要 · Abstract (English)
Diffusion large language models (dLLMs) generate text by iteratively denoising masked token sequences. Although dLLMs can predict all masked positions in parallel within each step, the large number of denoising iterations still makes inference expensive. This cost can be reduced spatially by unmasking multiple tokens per step, or temporally by collapsing multiple denoising steps into one verification call. We propose Parallel Speculative Decoding (PSD), a training-free framework that jointly improves inference along both axes. Using the confidence scores from a single forward pass, PSD selects positions to unmask via a configurable, adaptive unmasking policy and constructs multi-depth speculative drafts without extra model calls. A final batched verification pass then applies hierarchical acceptance, keeping the deepest draft that remains consistent with the updated predictions. Experiments on three dLLMs across reasoning and code generation tasks show that PSD achieves favorable trade-offs between inference efficiency and generation quality, reaching up to $5.5\times$ tokens per forward pass with accuracy comparable to greedy decoding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。