通过前瞻并行解码，将扩散语言模型推理速度提升至每轮10个以上词元。
LoPA: Scaling dLLM Inference via Lookahead Parallel Decoding
- 提出无需训练的前瞻并行解码方法，动态选择最优词元生成顺序。
- 在GSM8K上使D2F-Dream模型每轮推理达10.1个词元，效率显著提升。
- 适合追求高吞吐量推理的开发者与部署场景，尤其适用于多卡环境。
扩散大语言模型（dLLMs）展现出高速推理的巨大潜力。然而，现有基于置信度的解码策略受限于并行度，通常每前向传播仅能生成1–3个词元（TPF）。本文发现，dLLM推理中的并行度高度依赖于词元填充顺序（TFO）。为此，我们提出无需训练、即插即用的前瞻并行解码（LoPA）算法，通过并行分支探索不同候选TFO，根据分支置信度选择未来并行潜力最高的顺序。将LoPA应用于当前最优的D2F模型，显著提升解码效率：在GSM8K数据集上，D2F-Dream的TPF达到10.1，且性能优于基线Dream。为进一步支持这种前所未有的并行度，我们设计了具备分支并行（BP）特性的专用多设备推理系统，在多GPU部署下实现单样本每秒1073.9词元的吞吐量。代码已公开于https://github.com/zhijie-group/LoPA。
原文摘要 · Abstract (English)
Diffusion Large Language Models (dLLMs) have demonstrated significant potential for high-speed inference. However, current confidence-driven decoding strategies are constrained by limited parallelism, typically achieving only 1--3 tokens per forward pass (TPF). In this work, we identify that the degree of parallelism during dLLM inference is highly sensitive to the Token Filling Order (TFO). Then, we introduce Lookahead PArallel Decoding LoPA, a training-free, plug-and-play algorithm, to identify a superior TFO and hence accelerate inference. LoPA concurrently explores distinct candidate TFOs via parallel branches, and selects the one with the highest potential for future parallelism based on branch confidence. We apply LoPA to the state-of-the-art D2F model and observe a substantial enhancement in decoding efficiency. Notably, LoPA increases the TPF of D2F-Dream to 10.1 on the GSM8K while maintaining performance superior to the Dream baseline. Furthermore, to facilitate this unprecedented degree of parallelism, we develop a specialized multi-device inference system featuring Branch Parallelism (BP), which achieves a single-sample throughput of 1073.9 tokens per second under multi-GPU deployment. The code is available at https://github.com/zhijie-group/LoPA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。