通过追踪早期预测提升生成速度，让扩散模型更快更准。
CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit
- 用历史预测证据量化令牌解码潜力，避免重复掩码。
- 在8个基准上最高提速5.48倍，大模型下准确率还升0.48。
- 无需训练，适配多种模型和长文本，可与主流优化共存。
扩散语言模型通过迭代去噪生成文本。现有并行解码方法每步仅确认高置信度位置，其余重新掩码。分析去噪轨迹发现：模型常提前多步就预测正确词元，但需等待置信度达标才解码，导致已正确词元反复被掩码，造成冗余迭代。为此提出Trace Credit，通过累积历史证据量化词元解码潜力。基于此设计CreditDecoding，无需训练，融合历史信用与当前逻辑值，提升正确但低置信词元的置信度，加速去噪并增强鲁棒性。在8个基准上，该方法实现最高5.48倍加速，LLaDA-8B上准确率提升0.48，并在不同架构与参数规模下持续改进。支持长上下文，且与主流推理优化正交，具有广泛实用性。
原文摘要 · Abstract (English)
Diffusion large language models (dLLMs) generate text through iterative denoising. In commonly adopted parallel decoding schemes, each step confirms only high-confidence positions while remasking the others. By analyzing dLLM denoising traces, we uncover a key inefficiency: models often predict the correct target token several steps before its confidence becomes high enough to be decoded. This gap between early prediction and late decoding forces repeated remasking of already-correct tokens, causing redundant iterations and limiting acceleration. To exploit this temporal redundancy, we introduce Trace Credit to quantify a token's decoding potential by accumulating historical evidence. Building on this, we propose CreditDecoding, a training-free parallel decoding method that fuses Trace Credit with current logits to boost the confidence of correct but underconfident tokens, thereby accelerating denoising and improving robustness. On eight benchmarks, CreditDecoding achieves up to 5.48 times speedup with +0.48 accuracy on LLaDA-8B and consistently improves performance across diverse dLLM architectures and parameter scales. It further scales to long contexts and remains orthogonal to mainstream inference optimizations, making it a practical and widely applicable solution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。