提出无维度误差界,实现高维生成模型的可靠分析。
Dimension-free error estimate for diffusion model and optimal scheduling
- 用光滑测试函数定义新度量,突破传统方法的维度限制。
- 导出可量化生成样本与真实数据差距的无维度误差上界。
- 推导最优采样时序调度策略,为现有方法提供理论支持。
扩散生成模型通过模拟奥恩斯坦-乌伦贝克(OU)过程的逆时间演化来生成合成数据。由于关联的得分函数通常未知,需用训练好的神经网络近似。该近似以及有限时间模拟、离散化和统计近似引入多重误差源,其对生成样本的影响需被精确理解。以往分析多以Wasserstein距离或KL散度衡量误差,但前者在高维下边界随维度恶化,后者要求分布绝对连续。本文首次建立生成分布与真实分布间差异的显式、无维度误差上界,采用具有有界一阶与二阶导数的光滑测试函数作为度量标准。核心创新在于使用此弱形式度量获得维度无关的保证,代价是测试函数需更高正则性。作为应用,我们提出并求解一个变分问题以最小化时间离散误差,从而推导出逆向扩散过程的最优时间调度策略。有趣的是,该调度策略曾出现在文献中不同背景下;本工作为其最优性提供了全新理论依据,即基于最小化生成采样中的离散化偏差。
原文摘要 · Abstract (English)
Diffusion generative models have emerged as powerful tools for producing synthetic data from an empirically observed distribution. A common approach involves simulating the time-reversal of an Ornstein-Uhlenbeck (OU) process initialized at the true data distribution. Since the score function associated with the OU process is typically unknown, it is approximated using a trained neural network. This approximation, along with finite time simulation, time discretization and statistical approximation, introduce several sources of error whose impact on the generated samples must be carefully understood. Previous analyses have quantified the error between the generated and the true data distributions in terms of Wasserstein distance or Kullback-Leibler (KL) divergence. However, both metrics present limitations: KL divergence requires absolute continuity between distributions, while Wasserstein distance, though more general, leads to error bounds that scale poorly with dimension, rendering them impractical in high-dimensional settings. In this work, we derive an explicit, dimension-free bound on the discrepancy between the generated and the true data distributions. The bound is expressed in terms of a smooth test functional with bounded first and second derivatives. The key novelty lies in the use of this weaker, functional metric to obtain dimension-independent guarantees, at the cost of higher regularity on the test functions. As an application, we formulate and solve a variational problem to minimize the time-discretization error, leading to the derivation of an optimal time-scheduling strategy for the reverse-time diffusion. Interestingly, this scheduler has appeared previously in the literature in a different context; our analysis provides a new justification for its optimality, now grounded in minimizing the discretization bias in generative sampling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。