arXiv:2512.00580cs.LGstat.ML2025-12被引 9

首次给出离散扩散模型的非渐近收敛分析,突破维度与数据分布限制。

Non-Asymptotic Convergence of Discrete Diffusion Models: Masked and Random Walk dynamics

  • 针对三种离散扩散模型设计欧拉型离散化方法
  • 在无界得分估计下实现线性维度复杂度的收敛保证
  • 适用于有限状态与可数无穷空间,适合理论研究者

基于高斯加噪过程的连续状态扩散模型在理论和实践上已相对成熟。相比之下,离散状态空间上的扩散模型研究仍不充分,尤其因其组合结构及在生成建模中较新引入而面临挑战。本文为三种流行的离散扩散模型(DDMs)建立了新的、精确的收敛保证:两种用于有限状态空间,分别基于随机游走和掩码过程;第三种定义在可数无限空间 $ ^d$,使用漂移随机游走作为前向过程。尽管后向过程可通过离散得分函数刻画且理论上可估计,但精确模拟不可行,需依赖时间离散化。本文研究欧拉型近似,建立在极小假设下关于数据分布的KL散度与总变差距离的收敛界。据我们所知,该工作首次提供无需对估计得分施加有界性假设的最优非渐近收敛保证,且每种方法的计算复杂度仅随维度线性增长(含对数因子)。

原文摘要 · Abstract (English)

Diffusion models for continuous state spaces based on Gaussian noising processes are now relatively well understood from both practical and theoretical perspectives. In contrast, results for diffusion models on discrete state spaces remain far less explored and pose significant challenges, particularly due to their combinatorial structure and their more recent introduction in generative modelling. In this work, we establish new and sharp convergence guarantees for three popular discrete diffusion models (DDMs). Two of these models are designed for finite state spaces and are based respectively on the random walk and the masking process. The third DDM we consider is defined on the countably infinite space $\mathbb{N}^d$ and uses a drifted random walk as its forward process. For each of these models, the backward process can be characterized by a discrete score function that can, in principle, be estimated. However, even with perfect access to these scores, simulating the exact backward process is infeasible, and one must rely on time discretization. In this work, we study Euler-type approximations and establish convergence bounds in both Kullback-Leibler divergence and total variation distance for the resulting models, under minimal assumptions on the data distribution. To the best of our knowledge, this study provides the optimal non-asymptotic convergence guarantees for these noising processes that do not rely on boundedness assumptions on the estimated score. In particular, the computational complexity of each method scales only linearly in the dimension, up to logarithmic factors.

扩散模型离散生成理论分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。