让扩散语言模型更高效，通过动态聚焦可解码的词元
FOCUS: DLLMs Know How to Tame Their Compute Bound
- 根据注意力重要性预测词元是否可解码，动态分配计算资源
- 在大批次场景下，推理吞吐量最高提升3.52倍，生成质量不降反升
- 适合需要高并发、低延迟的扩散模型部署场景
扩散型大语言模型（DLLMs）为自回归模型提供了有吸引力的替代方案，但其部署受限于高昂的解码成本。本文发现DLLM解码中的关键效率瓶颈：尽管计算在词元块上并行，但每个扩散步骤仅少数词元可解码，导致大量计算被浪费在不可解码的词元上。我们进一步观察到，注意力生成的词元重要性与词元解码概率存在强相关性。基于此，提出FOCUS推理系统，通过动态聚焦可解码词元、实时剔除不可解码词元，提升有效批处理规模，缓解计算瓶颈，实现可扩展的吞吐量。实验表明，在大批次设置下，FOCUS相比生产级引擎LMDeploy最高实现3.52倍吞吐量提升，且在多个基准测试中保持或改善生成质量。
原文摘要 · Abstract (English)
Diffusion Large Language Models (DLLMs) offer a compelling alternative to Auto-Regressive models, but their deployment is constrained by high decoding cost. In this work, we identify a key inefficiency in DLLM decoding: while computation is parallelized over token blocks, only a small subset of tokens is decodable at each diffusion step, causing most compute to be wasted on non-decodable tokens. We further observe a strong correlation between attention-derived token importance and token-wise decoding probability. Based on this insight, we propose FOCUS, an inference system designed for DLLMs. By dynamically focusing computation on decodable tokens and evicting non-decodable ones on-the-fly, FOCUS increases the effective batch size, alleviating compute limitations and enabling scalable throughput. Empirical evaluations demonstrate that FOCUS achieves up to 3.52$\times$ throughput improvement over the production-grade engine LMDeploy in large-batch settings, while preserving or improving generation quality across multiple benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。