提出C4框架,让扩散语言模型局部加速并全局提前终止。
Commit Locally, Exit Globally: Coordinating Adaptive Sampling and Early Exit in Diffusion Language Models

- 用独立门控分别控制每步提交和序列终止,解耦加速策略。
- 在12个零样本任务上减少64%–95%解码步骤,提速2.6–8.6倍。
- 适合追求高效推理的扩散语言模型研究与应用者。
扩散语言模型在每一步去噪过程中都会生成一个临时预测,许多任务中候选答案在此前已趋于稳定。这带来了两个加速机会:提前退出当前块和提前终止整个序列。但两者需不同判据,因块级加速是局部的,而序列终止是全局的,会冻结渐进式答案。现有方法通常只优化单一维度,且退出门依赖固定区域置信度或时序相关规则,而非候选答案本身。本文提出 $ extbf{C}^4$,通过为每个决策分配独立门控来协调两轴:$ extbf{C}$onfidence-Verified Early Exit (CVEE) 判定序列是否可终止,要求在每一步重新提取的候选跨度上具备置信度与一致的argmax稳定性;$ extbf{C}$ommit-$ extbf{C}$ore-Then-$ extbf{C}$onfirm (CCTC) 决定哪些词位置可提交,借用块内自回归冻结顺序:先锚定边界核心,再延迟一步确认其余位置,实现块加速而不允许答案内部局部提交影响序列终止。在使用 LLaDA 与 Dream 模型的12个零样本任务上,单一冻结配置下减少64%–95%解码步骤,端到端提速2.6至8.6倍。代码已开源:https://github.com/ming053l/C4-dLLM。
原文摘要 · Abstract (English)
Diffusion language models expose a provisional prediction at every denoising step, and on many tasks the candidate answer inside it stabilizes before the step schedule is exhausted. This creates two acceleration opportunities, leaving a block early and stopping the sequence early, but the two require different criteria because block acceleration is local whereas sequence termination is global and freezes the graded answer. Existing methods usually optimize only one axis, and existing exit gates rely on fixed-region confidence or schedule-dependent rules rather than the candidate answer itself. We present $\textbf{C}^4$, which coordinates the two axes by giving each decision its own gate. $\textbf{C}$onfidence-Verified Early Exit (CVEE) decides when the sequence may stop, requiring confidence and sustained argmax stability over a candidate span re-extracted at every step. $\textbf{C}$ommit-$\textbf{C}$ore-Then-$\textbf{C}$onfirm (CCTC) decides which token positions a step may commit by borrowing an autoregressive freezing order inside each block: it commits a boundary-anchored core and confirms deferred positions one step later, so the answer block can be accelerated without allowing local commits inside the answer span to determine sequence-level termination. On 12 zero-shot tasks with LLaDA and Dream, one frozen configuration removes 64--95% of decoding steps and delivers measured end-to-end speedups of 2.6 to 8.6 over full decoding. Code is available at https://github.com/ming053l/C4-dLLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。