arXiv:2607.03236cs.CL2026-07被引 2

让扩散语言模型更稳地生成文本,避免过早锁定错误选项。

TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding

论文配图:TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding
图 1 · 摘自论文原文
  • 基于历史预测轨迹动态判断何时该确定候选词
  • 在多个代码与数学任务上减少迭代步数并提升生成效率
  • 无需额外训练或计算,适合追求高效稳定的生成场景

扩散语言模型通过逐步去噪掩码位置生成文本,呈现出一系列预测分布的演化轨迹。现有解码器大多忽略这一轨迹,仅依据当前快照决定提交词语,将置信度与提交时机混淆:在上下文不完整时出现的瞬时最高概率可能被过早锁定,而跨步骤持续支持的候选反而延迟。本文提出无需训练的轨迹感知提交门控(TACG),将词元身份锚定于基础后验分布,仅用轨迹信号判断当前提议是否具备提交条件。TACG结合时间隐式对数几率引导(TILG),通过指数移动平均保留历史对数几率作为自参考,并在自然参数空间中对比当前与历史分布;辅以历史门控(HG)确保提交前短期提议一致性。配合上限预算的额外促进机制,实现无辅助网络、无需额外前向传播的稳定性约束提交规则。我们在LLaDA、Dream和LLaDA2-Mini上评估,涵盖代码(HumanEval、MBPP)与数学(GSM8K、MATH500)任务,结果表明其通常保持或提升准确率,同时减少去噪步数并提高每前向传播生成词数(TPF)。代码已公开于https://github.com/Clarence-CV/TACG-DLLM。

原文摘要 · Abstract (English)

Diffusion language models (DLLMs) generate text by iteratively denoising masked positions, exposing a trajectory of predictive distributions rather than a single instantaneous belief. Most existing decoders ignore this trajectory and commit tokens from the current snapshot alone, conflating confidence with commitment readiness: a transient top-1 peak under incomplete context can be locked in, while candidates with consistent cross-step support are delayed. We propose Trajectory-Aware Commit Gating (TACG), a training-free gate-level decoder that anchors token identities to the base posterior and uses trajectory-aware signals only to decide whether the current proposal is ready to commit. TACG combines Temporal Implicit Logits Guidance (TILG), which keeps an exponential moving average of past logits as a self-reference and contrasts the current logits against this reference in natural-parameter space, with a History Gate (HG) that enforces short-term proposal persistence before commitment. Together with a capped extra-promotion budget, these components yield a stability-constrained commit rule without auxiliary networks or extra forward passes. We evaluate TACG on LLaDA, Dream, and LLaDA2-Mini across code (HumanEval, MBPP) and math (GSM8K, MATH500) benchmarks; it typically improves or preserves accuracy while reducing denoising steps and increasing tokens per forward (TPF). The code is publicly available at https://github.com/Clarence-CV/TACG-DLLM.

扩散模型文本生成解码优化轨迹感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。