提出自适应并行解码，让扩散语言模型更快生成且不丢质量。
Accelerating Diffusion LLMs via Adaptive Parallel Decoding
- 动态调整并行生成的词元数量，结合扩散模型与小模型联合概率。
- 在多个下游任务上实现显著提速，质量损失极小，最高提升4.2倍吞吐。
- 适合追求高生成速度的场景，如实时对话系统或大规模文本生成。
LLM 的生成速度受自回归解码限制，即逐个预测词元。相比之下，扩散语言模型（dLLMs）理论上支持并行生成，但实际中难以在不显著降低质量的前提下达到自回归模型的速度。为此，我们提出自适应并行解码（APD），一种动态调节并行采样词元数量的新方法。通过将 dLLM 的边缘概率与一个小的辅助自回归模型下的序列联合概率进行乘积混合，实现对生成过程的调控。这与传统的推测解码相反，后者是用小模型起草、大模型验证。我们进一步优化了 APD，引入键值缓存机制并限制掩码输入大小。整体方法包含三个可调参数，灵活权衡吞吐与质量。实验表明，APD 在下游基准测试中实现了显著更高的吞吐量，同时仅带来微小的质量下降。
原文摘要 · Abstract (English)
The generation speed of LLMs are bottlenecked by autoregressive decoding, where tokens are predicted sequentially one by one. Alternatively, diffusion large language models (dLLMs) theoretically allow for parallel token generation, but in practice struggle to achieve the speed of autoregressive models without significantly sacrificing quality. We therefore introduce adaptive parallel decoding (APD), a novel method that dynamically adjusts the number of tokens sampled in parallel. We achieve this by defining a multiplicative mixture between the dLLM marginal probabilities and the joint probability of sequences under a small auxiliary autoregressive model. This inverts the standard setup of speculative decoding, where the goal is to sample from a large autoregressive verifier by drafting from a smaller model. We further optimize APD by enabling KV caching and limiting the size of the masked input. Altogether, our method puts forward three tunable parameters to flexibly tradeoff throughput and quality. We show that APD provides markedly higher throughput with minimal quality degradations on downstream benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。