解析引导如何精准调控掩码离散扩散模型的采样轨迹与分布形态
What Exactly Does Guidance Do in Masked Discrete Diffusion Models
- 基于无误差假设推导出引导反向动态的显式解，精确刻画引导机制
- 引导增强类别特异性区域，抑制共用区域，且强度越大越显著
- 发现1D/2D下引导导致不同协方差结构，且总变差衰减呈双指数级
我们研究了带有无分类器引导（CFG）的掩码离散扩散模型。在无得分误差和离散化误差的前提下，推导出引导反向动态的显式解，从而可精确刻画引导对采样行为的影响。当完整数据分布为多类混合，目标是特定类别采样时，引导会放大类特异性区域，抑制与其他类共享的区域。该效应依赖于引导强度 $w$，并导致采样分布中出现不同的协方差结构。值得注意的是，我们在1维和2维情况下观察到定量不同的行为。此外，我们证明当 $w$ 较大时，反向动态中总变差（TV）的衰减速率在1维和2维下均为关于 $w$ 的双指数级。这些发现揭示了引导不仅影响输出分布，还控制采样轨迹的动力学。理论分析得到实验验证，展示了引导的几何影响及其对收敛性的作用。
原文摘要 · Abstract (English)
We study masked discrete diffusion models with classifier-free guidance (CFG). Assuming no score error nor discretization error, we derive an explicit solution to the guided reverse dynamics, so that how guidance influences the sampling behavior can be precisely characterized. When the full data distribution is a mixture over classes and the goal is to sample from a specific class, guidance amplifies class-specific regions while suppresses regions shared with other classes. This effect depends on the guidance strength $w$ and induces distinct covariance structures in the sampled distribution. Notably, we observe quantitatively different behaviors in $1$D and $2$D. We also show that for large $w$, the decay rate of the total variation ($\mathrm{TV}$) along the reverse dynamics is double-exponential in $w$ for both $1$D and $2$D. These findings highlight the role of guidance, not just in shaping the output distribution, but also in controlling the dynamics of the sampling trajectory. Our theoretical analysis is supported by experiments that illustrate the geometric effects of guidance and its impact on convergence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。