提出高阶微分方程求解器的收敛性理论,为扩散模型采样提供精度保证。
Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models
- 基于带约束的评分函数,设计高阶龙格-库塔求解器。
- 证明生成分布与目标分布的总变差距离上界为O(d^{7/4}ε_{score}^{1/2} + d(dH_max)^p)。
- 理论适用于任意方差调度,适合关注采样可靠性与效率的研究者。
扩散概率模型通过学习逆转噪声注入过程生成样本。关键进展是将逆向采样过程重构成确定性的概率流常微分方程(ODE),从而可使用高阶数值求解器实现高效采样。不同于传统时间积分分析,该采样精度不仅取决于数值积分误差,还受学习到的评分函数近似质量与光滑性及其相互作用的影响。本文对具有任意方差调度的一般前向过程,给出了由概率流ODE导出的确定性采样器的严格收敛性分析。具体地,我们在学习评分函数一阶与二阶导数有界的实用假设下,研究了p阶(指数)龙格-库塔格式。我们证明生成分布与目标分布之间的总变差距离可被控制在 $ Oigl(d^{rac{7}{4}}\varepsilon_{ ext{score}}^{rac{1}{2}} +d(dH_{ ext{max}})^pigr) $,其中 $\varepsilon^2_{ ext{score}}$ 表示评分函数近似的 $L^2$ 误差,$d$ 为数据维度,$H_{ ext{max}}$ 为最大求解器步长。基准数据集上的数值实验进一步验证了学习到的评分函数导数在实践中是有界的。
原文摘要 · Abstract (English)
Diffusion probabilistic models generate samples by learning to reverse a noise-injection process that transforms data into noise. A key development is the reformulation of the reverse sampling process as a deterministic probability flow ordinary differential equation (ODE), which allows for efficient sampling using high-order numerical solvers. Unlike traditional time integrator analysis, the accuracy of this sampling procedure depends not only on numerical integration errors but also on the approximation quality and regularity of the learned score function, as well as their interaction. In this work, we present a rigorous convergence analysis of deterministic samplers derived from probability flow ODEs for general forward processes with arbitrary variance schedules. Specifically, we develop and analyze $p$-th order (exponential) Runge-Kutta schemes, under the practical assumption that the first and second derivatives of the learned score function are bounded. We prove that the total variation distance between the generated and target distributions can be bounded as \begin{align*} O\bigl(d^{\frac{7}{4}}\varepsilon_{\text{score}}^{\frac{1}{2}} +d(dH_{\max})^p\bigr), \end{align*} where $\varepsilon^2_{\text{score}}$ denotes the $L^2$ error in the score function approximation, $d$ is the data dimension, and $H_{\max}$ represents the maximum solver step size. Numerical experiments on benchmark datasets further confirm that the derivatives of the learned score function are bounded in practice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。