首次在离散空间中证明了得分扩散模型的收敛性，为理论基础提供关键支撑。
Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis
- 基于连续时间马尔可夫链框架，设计离散时间采样算法
- 在合理假设下，得到KL散度与总变差距离的收敛界
- 结果接近最优，适合关注理论严谨性的研究者
扩散模型在高维样本生成中取得显著成功。尽管连续状态扩散模型的理论保障已广泛研究，其离散状态版本的收敛性分析仍不充分。本文在连续时间马尔可夫链（CTMC）框架下，研究得分型离散扩散模型的理论性质。提出一种在一般状态空间 $[S]^d$ 上的离散时间采样算法，利用预设时间点的得分估计器进行采样。在合理假设下，推导出生成分布与数据分布之间KL散度和总变差（TV）距离的收敛界，涵盖有无提前停止的情形。值得注意的是，我们的KL散度界在维度 $d$ 上近乎线性，与当前扩散模型最优结果一致。分析采用基于Girsanov的方法，揭示了离散得分函数的关键性质，这些性质对刻画离散时间采样过程至关重要。
原文摘要 · Abstract (English)
Diffusion models have achieved great success in generating high-dimensional samples across various applications. While the theoretical guarantees for continuous-state diffusion models have been extensively studied, the convergence analysis of the discrete-state counterparts remains under-explored. In this paper, we study the theoretical aspects of score-based discrete diffusion models under the Continuous Time Markov Chain (CTMC) framework. We introduce a discrete-time sampling algorithm in the general state space $[S]^d$ that utilizes score estimators at predefined time points. We derive convergence bounds for the Kullback-Leibler (KL) divergence and total variation (TV) distance between the generated sample distribution and the data distribution, considering both scenarios with and without early stopping under reasonable assumptions. Notably, our KL divergence bounds are nearly linear in the dimension $d$, aligning with state-of-the-art results for diffusion models. Our convergence analysis employs a Girsanov-based method and establishes key properties of the discrete score function, which are essential for characterizing the discrete-time sampling process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。