arXiv:2602.22505cs.LGstat.ML2026-02被引 10

首次为掩码扩散模型提供紧致收敛分析,揭示采样器真实误差边界。

Sharp Convergence Rates for Masked Diffusion Models

  • 基于总变差直接分析欧拉采样器,降低对得分估计的依赖
  • 证明欧拉采样器收敛率与维度和精度紧密相关,首次给出下界
  • 发现FHS采样器误差仅来自得分估计,且该误差不可再减

离散扩散模型在文本等符号领域表现优异,其中掩码(吸收率)变体已成为自回归模型的有力竞争者。当前主流采样器如欧拉方法,近期出现的首次命中采样器(FHS)也展现出显著潜力。然而,这些方法的理论理解仍不充分:现有分析多基于KL散度,参数依赖松散且需强假设;同时未覆盖高性能的FHS。本文首次构建基于总变差(TV)的分析框架,突破上述限制——对欧拉方法,放松得分估计假设,改善参数依赖性,并在无需代理初始化下建立收敛性;进一步给出首个欧拉采样的收敛下界,与数据维度 $d$ 及目标精度 $\varepsilon$ 的紧致关系一致。对FHS,证明其采样误差仅由得分估计引起,且该误差具有匹配的下界。整体分析引入沿连续时间马尔可夫链轨迹的直接TV误差分解与路径级解耦分析,或具独立研究价值。

原文摘要 · Abstract (English)

Discrete diffusion models have achieved strong empirical performance in text and other symbolic domains, with masked (absorbing-rate) variants emerging as competitive alternatives to autoregressive models. Among existing samplers, the Euler method remains the standard choice in many applications, and more recently, the First-Hitting Sampler (FHS) has shown considerable promise for masked diffusion models. Despite their practical success, the theoretical understanding of these samplers remains limited. Existing analyses are conducted in Kullback-Leibler (KL) divergence, which often yields loose parameter dependencies and requires strong assumptions on score estimation. Moreover, these guarantees do not cover recently developed high-performance sampler of FHS. In this work, we first develop a direct total-variation (TV) based analysis for the Euler method that overcomes these limitations. Our results relax assumptions on score estimation, improve parameter dependencies, and establish convergence guarantees without requiring any surrogate initialization. Also for this setting, we provide the first convergence lower bound for the Euler sampler, establishing tightness with respect to both the data dimension $d$ and the target accuracy $\varepsilon$. Finally, we analyze the FHS sampler and show that it incurs no sampling error beyond that induced by score estimation, which we show to be tight with a matching lower error bound. Overall, our analysis introduces a direct TV-based error decomposition along the CTMC trajectory and a decoupling-based path-wise analysis for FHS, which may be of independent interest.

扩散模型采样器分析收敛率概率推断

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。