arXiv:2509.26432cs.LGcs.AI2025-09被引 27

动态调整解码块大小,提升扩散模型生成准确率

AdaBlock-dLLM: Semantic-Aware Diffusion LLM Inference via Adaptive Block Size

  • 根据语义置信度波动自适应调整解码块大小
  • 相同吞吐下准确率最高提升5.3%
  • 无需训练,可直接部署于现有扩散语言模型

基于扩散的大型语言模型(dLLMs)因其固有的并行解码能力,正成为自回归模型的有力替代。在多种解码策略中,分块半自回归(semi-AR)方法因支持键值缓存且在精度与速度间取得良好平衡而被广泛采用。然而本文指出,传统固定块大小的semi-AR解码存在两大根本缺陷:一是高置信度令牌在当前块外被延迟解码,二是低置信度令牌过早提交导致错误。本文首次系统性地挑战了固定块大小设定。通过对去噪过程中置信度动态的统计分析,我们识别出一个编码局部语义结构的波动带(VB)区域,可用于指导自适应块尺寸调整。基于此,提出AdaBlock-dLLM——一种无需训练、即插即用的运行时调度器,通过动态调整块大小,使解码边界与语义步长对齐。跨多个基准的大量实验表明,在相同吞吐预算下,AdaBlock-dLLM最高可实现5.3%的准确率提升。该研究不仅为推理优化提供新思路,也为未来dLLM的训练策略提供启发。代码已开源。

原文摘要 · Abstract (English)

Diffusion-based large language models (dLLMs) are gaining attention for their inherent capacity for parallel decoding, offering a compelling alternative to autoregressive LLMs. Among various decoding strategies, block-wise semi-autoregressive (semi-AR) approaches are widely adopted due to their support for KV caching and their favorable accuracy-speed trade-off. However, this paper identifies two fundamental limitations in the conventional semi-AR decoding approach that applies a fixed block size: i) late decoding overhead, where the unmasking of high-confidence tokens outside the current block is unnecessarily delayed, and ii) premature decoding error, where low-confidence tokens inside the current block are committed too early, leading to incorrect tokens. This paper presents the first systematic investigation challenging the fixed block size setting in semi-AR decoding. Through a statistical analysis of confidence dynamics during the denoising process, we identify a volatility band (VB) region during dLLM decoding, which encodes local semantic structure and can be used to guide adaptive block sizing. Leveraging these insights, we introduce AdaBlock-dLLM, a training-free, plug-and-play scheduler that adaptively aligns block boundaries with semantic steps by adjusting block size during runtime. Extensive experiments across diverse benchmarks show that AdaBlock-dLLM achieves up to 5.3% accuracy improvement under the same throughput budget. Beyond inference-time optimization, we hope our semantics-aware adaptive scheduling approach and confidence-based analysis will inspire future training strategies for dLLMs. Our code is available at https://github.com/lgxi24/AdaBlock-dLLM.

扩散模型自回归解码优化语义感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。