发现引导无需全程开启,提前决策可提升生成效率与稳定性。
Commitment Before Realization: When Classifier-Free Guidance Becomes Unnecessary in Masked Diffusion Language Models

- 根据未来成功概率判断何时可停用引导,实现动态决策
- 多数提示词在早期即可确定最终成败,后期引导无效甚至有害
- 适合追求高效、稳定生成的场景,尤其在长文本生成中
分类器无关引导(CFG)通常在掩码扩散语言模型解码中全程启用,但其收益随提示词和时间变化。本文通过比较部分输出后继续使用CFG与仅用基础模型的最终约束满足概率,定义了引导的剩余价值。结果显示,引导依赖高度依赖提示词;许多提示词无需引导即成功,而某些提示词中引导无益甚至有害,且收益多集中于早期。定义承诺时点 $\astar$ 为从该点起切换至基础模型,最终成功率下降不超过容忍度的最早时刻。基础模型下的成功概率(即承诺值)是鞅。一阶近似下,每步引导效果由引导逻辑方向与后续承诺值的协方差决定。在13个子任务上,按各自交叉拟合的承诺时点冻结,性能不低于全程使用CFG,即使仍有大量标记未填充。该边界还识别出并行度更高但约束成功率损失微小的区域,尽管流畅性随并行宽度下降。对失败轨迹,重新开放已承诺位置可有效提升恢复能力。
原文摘要 · Abstract (English)
Classifier-free guidance (CFG) is usually kept on throughout masked diffusion language model decoding, although its benefit varies across prompts and over time. We study when CFG is actually needed by comparing, from any partial output, the probability of eventual constraint satisfaction under continued CFG and under base-only continuation. Their difference defines the remaining value of guidance. Guidance dependence is highly prompt-specific. Many prompts already succeed without CFG, while for others it provides no measurable benefit or can be harmful. For prompts that do benefit, the gain is often concentrated early. We define the commitment horizon $\astar$ as the earliest point from which switching all remaining decoding to the base model reduces final success by no more than a chosen tolerance. Under the base model, the corresponding success probability, or committor, is a martingale. To first order, CFG's per-step effect is governed by the covariance between the guidance logit direction and the successor committor. This gives a local account of when guidance can help, but it does not by itself locate the horizon. Among prompts with an observed preterminal horizon, $\astar$ is usually early and varies more within constraint families than between them. Freezing each prompt at its own cross-fitted horizon is noninferior to full CFG on all 13 subtasks at the prespecified margin, even while many tokens remain masked. This separates commitment from realization. The boundary also identifies a later region in which higher parallelism adds only a small cost in constraint success, although fluency still degrades with parallel width. For failed trajectories, reopening committed positions improves recovery in both failure modes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。