arXiv:2509.25188cs.CL2025-09被引 23

让大模型自动决定哪些词能并行生成,速度提升超20倍且不丢质量

Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding

  • 训练轻量模型动态判断每个词是否该并行输出
  • 实测最高提速22.58倍,结合缓存可达57.51倍
  • 适合需要高速推理的部署场景,如对话系统

自回归解码在大语言模型中需耗时$/mathcal{O}(n)$完成$n$个词,严重限制推理吞吐。基于扩散的LLM(dLLMs)通过迭代去噪实现并行生成,但现有策略依赖固定、与输入无关的启发式规则(如置信度阈值),无法适应不同输入特性,导致跨任务速度-质量权衡不佳。本文提出学习并行解码(Learn2PD),训练一个轻量自适应过滤模型,预测每个词位当前预测是否等于最终输出。该模型逼近理想并行解码策略——仅在预测正确时解码。过滤器以微调方式训练,优化仅需分钟级GPU时间。此外引入结尾文本预测(EoTP)识别序列结束,避免冗余填充解码。在LLaDA基准测试中,本方法在不损失性能下实现最高22.58倍加速,结合KV-Cache可达57.51倍。

原文摘要 · Abstract (English)

Autoregressive decoding in large language models (LLMs) requires $\mathcal{O}(n)$ sequential steps for $n$ tokens, fundamentally limiting inference throughput. Recent diffusion-based LLMs (dLLMs) enable parallel token generation through iterative denoising. However, current parallel decoding strategies rely on fixed, input-agnostic heuristics (e.g., confidence thresholds), which fail to adapt to input-specific characteristics, resulting in suboptimal speed-quality trade-offs across diverse NLP tasks. In this work, we explore a more flexible and dynamic approach to parallel decoding. We propose Learning to Parallel Decode (Learn2PD), a framework that trains a lightweight and adaptive filter model to predict, for each token position, whether the current prediction matches the final output. This learned filter approximates an oracle parallel decoding strategy that unmasks tokens only when correctly predicted. Importantly, the filter model is learned in a post-training manner, requiring only a small amount of computation to optimize it (minute-level GPU time). Additionally, we introduce End-of-Text Prediction (EoTP) to detect decoding completion at the end of sequence, avoiding redundant decoding of padding tokens. Experiments on the LLaDA benchmark demonstrate that our method achieves up to 22.58$\times$ speedup without any performance drop, and up to 57.51$\times$ when combined with KV-Cache.

扩散模型推理加速并行生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。