提出扩散模型的实例自适应收敛理论,精度更高且适用性更广。
Instance-dependent Convergence Theory for Diffusion Models
- 基于目标分布平滑性设计自适应收敛率,提升理论精度。
- 收敛复杂度达 $\min\{d,d^{2/3}L^{1/3},d^{1/3}L\}\varepsilon^{-2/3}$,优于传统全局界。
- 在高斯混合模型中,$L$ 仅对数增长,适合高维复杂分布建模者。
基于得分的扩散模型在机器学习与人工智能中展现出卓越的生成性能,尤其擅长从复杂概率分布中生成高质量样本。提升扩散模型的理论理解,尤其是收敛性分析,受到广泛关注。本文提出一种依赖于具体样本的收敛率,称为实例自适应界。具体而言,建立了迭代复杂度为 $\min\{d,d^{2/3}L^{1/3},d^{1/3}L\}\varepsilon^{-2/3}$(含对数因子)的上界,其中 $d$ 为数据维度,$\varepsilon$ 表示输出精度(以总变差距离衡量)。此外,$L$ 为松弛的利普希茨常数,在高斯混合模型中其增长仅与分量数量、维度和迭代次数呈对数关系,展现出广泛适用性。
原文摘要 · Abstract (English)
Score-based diffusion models have demonstrated outstanding empirical performance in machine learning and artificial intelligence, particularly in generating high-quality new samples from complex probability distributions. Improving the theoretical understanding of diffusion models, with a particular focus on the convergence analysis, has attracted significant attention. In this work, we develop a convergence rate that is adaptive to the smoothness of different target distributions, referred to as instance-dependent bound. Specifically, we establish an iteration complexity of $\min\{d,d^{2/3}L^{1/3},d^{1/3}L\}\varepsilon^{-2/3}$ (up to logarithmic factors), where $d$ denotes the data dimension, and $\varepsilon$ quantifies the output accuracy in terms of total variation (TV) distance. In addition, $L$ represents a relaxed Lipschitz constant, which, in the case of Gaussian mixture models, scales only logarithmically with the number of components, the dimension and iteration number, demonstrating broad applicability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。