解析高斯分布下扩散采样误差来源,揭示数据特性与算法参数的精细关系。
From Score Matching to Diffusion: A Fine-Grained Error Analysis in the Gaussian Setting
- 在高斯设定下精确分解采样误差的四大来源:得分匹配与扩散过程中的泛化、优化、离散化及噪声幅度误差。
- 推导出Wasserstein误差的显式表达式,其形式为数据功率谱的核范数,依赖于算法参数。
- 结果为优化采样精度提供了理论依据,适合研究生成模型误差机制的学者参考。
从仅通过离散样本访问的未知分布中进行采样,是生成式人工智能的核心问题。当前最先进方法采用两步流程:先估计得分函数(平滑对数分布的梯度),再使用基于扩散的采样算法(如Langevin或扩散模型)。最终分布的准确性受四个主要因素影响:得分匹配中的泛化与优化误差,以及扩散过程中的离散化误差和最小噪声幅度。本文在高斯设定下显式分析了使用扩散采样器时的采样误差,提供了针对Wasserstein采样误差的精细分析。该分析揭示了数据分布各向异性(由其功率谱编码)如何与端到端采样方法的关键参数(包括初始样本数、得分匹配与扩散中的步长、噪声幅度)相互作用。特别地,我们证明了Wasserstein采样误差可表示为数据功率谱的核型范数,其中具体核函数取决于方法参数。该结果为进一步分析采样精度优化中的权衡提供了理论基础。
原文摘要 · Abstract (English)
Sampling from an unknown distribution, accessible only through discrete samples, is a fundamental problem at the core of generative AI. The current state-of-the-art methods follow a two-step process: first, estimating the score function (the gradient of a smoothed log-distribution) and then applying a diffusion-based sampling algorithm -- such as Langevin or Diffusion models. The resulting distribution's correctness can be impacted by four major factors: the generalization and optimization errors in score matching, and the discretization and minimal noise amplitude in the diffusion. In this paper, we make the sampling error explicit when using a diffusion sampler in the Gaussian setting. We provide a sharp analysis of the Wasserstein sampling error that arises from these four error sources. This allows us to rigorously track how the anisotropy of the data distribution (encoded by its power spectrum) interacts with key parameters of the end-to-end sampling method, including the number of initial samples, the stepsizes in both score matching and diffusion, and the noise amplitude. Notably, we show that the Wasserstein sampling error can be expressed as a kernel-type norm of the data power spectrum, where the specific kernel depends on the method parameters. This result provides a foundation for further analysis of the tradeoffs involved in optimizing sampling accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。