不训练即可提升扩散语言模型推理速度，靠动态调整生成步长和采样范围。
Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration
- 根据未掩码词的置信度动态调整生成块大小和步长。
- 在保持精度的前提下，推理吞吐量提升1.1至2.28倍。
- 无需训练、兼容现有缓存结构，适合快速部署到主流扩散模型。
我们提出CadLLM，一种无需训练的方法，用于加速基于扩散的大型语言模型（dLLMs）的推理吞吐量。我们首先研究了不同块和步骤间词元去掩码置信度的动态特性。基于此观察，提出一种轻量级自适应方法，根据未掩码词元的平均置信度动态控制生成块大小、步长及阈值。此外，通过动态利用词汇表子集来调节采样广度，进一步降低softmax开销。CadLLM是一种即插即用、模型无关的方法，兼容基于键值缓存的dLLMs。在四个主流任务上的大量实验表明，相较于当前最优基线，CadLLM在保持竞争力准确率的同时，实现了最高达1.1–2.28倍的吞吐量提升。
原文摘要 · Abstract (English)
We present CadLLM, a training-free method to accelerate the inference throughput of diffusion-based LLMs (dLLMs). We first investigate the dynamic nature of token unmasking confidence across blocks and steps. Based on this observation, we present a lightweight adaptive approach that controls the generation block size, step size, and threshold based on the average confidence of unmasked tokens. We further reduce softmax overhead by dynamically leveraging a subset of the vocabulary to regulate sampling breadth. CadLLM is a plug-and-play, model-agnostic method compatible with KV-cache-based dLLMs. Extensive experiments on four popular tasks demonstrate that CadLLM yields up to 1.1-2.28x throughput improvement over the state-of-the-art baseline with competitive accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。