证明扩散模型在高维流形上可实现与维度无关的收敛速度
Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions
- 基于高维数据流形假设，构建新理论框架连接扩散模型与高斯过程极值理论
- 证明得分学习和采样复杂度的收敛速率均不依赖于环境维度
- 为扩散模型在高维数据生成中的成功提供理论解释，适合理论研究者
去噪扩散概率模型（DDPM）是生成高维数据分布合成数据的强大前沿方法，广泛应用于图像、音频、视频生成及科学领域。曼达伯假说认为高维数据通常位于低维流形上，这一观点在大量实例中被广泛接受。尽管已有研究提供了关于扩散模型如何适应曼达伯假说的重要洞见，但尚未完全解释其卓越的实证表现，因此该方向极具研究价值。本文在曼达伯假设下研究DDPM，证明其在得分学习方面达到与环境维度无关的收敛速率；在采样复杂度方面，相对于Wasserstein距离也获得与维度无关的速率。为此，我们建立了一个新框架，将扩散模型与广义高斯过程极值理论相联系。
原文摘要 · Abstract (English)
Denoising Diffusion Probabilistic Models (DDPM) are powerful state-of-the-art methods used to generate synthetic data from high-dimensional data distributions and are widely used for image, audio, and video generation as well as many more applications in science and beyond. The \textit{manifold hypothesis} states that high-dimensional data often lie on lower-dimensional manifolds within the ambient space, and is widely believed to hold in provided examples. While recent results have provided invaluable insight into how diffusion models adapt to the manifold hypothesis, they do not capture the great empirical success of these models, making this a very fruitful research direction. In this work, we study DDPMs under the manifold hypothesis and prove that they achieve rates independent of the ambient dimension in terms of score learning. In terms of sampling complexity, we obtain rates independent of the ambient dimension w.r.t.\ the Wasserstein distance. We do this by developing a new framework connecting diffusion models to the well-studied theory of extrema of Gaussian Processes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。