用高阶方法加速扩散模型采样，无需重训练
Faster Diffusion Models via Higher-Order Approximation
- 基于高阶微分方程求解思路，用插值和迭代提升采样效率
- 采样复杂度仅需 $ d^{1+2/K} \< epsilon^{-1/K} $ 次得分函数计算
- 对得分估计误差鲁棒，适合追求高效采样的研究者
本文在不进行额外训练的前提下，研究了扩散模型的可证明加速。针对在 $\mathbb{R}^d$ 中以总变差距离 $\varepsilon$ 内逼近目标数据分布的任务，提出一种无需训练的采样算法。该算法在得分准确时，仅需约 $ d^{1+2/K} \varepsilon^{-1/K} $ 次得分函数评估（含对数因子），其中 $K>0$ 为任意固定整数。该方法适用于广泛的分布类型，无需平滑性或对数凹性假设。理论对得分估计误差具有鲁棒性，性能随误差增加而渐进下降，且不依赖于得分估计的高阶光滑性。算法受高阶常微分方程求解器启发，利用高阶拉格朗日插值与逐次精化来逼近概率流常微分方程的积分。本工作为理解高阶方法在加速采样中的有效性提供了理论框架。
原文摘要 · Abstract (English)
In this paper, we explore provable acceleration of diffusion models without any additional retraining. Focusing on the task of approximating a target data distribution in $\mathbb{R}^d$ to within $\varepsilon$ total-variation distance, we propose a principled, training-free sampling algorithm that requires only the order of $$ d^{1+2/K} \varepsilon^{-1/K} $$ score function evaluations (up to log factor) in the presence of accurate scores, where $K>0$ is an arbitrary fixed integer. This result applies to a broad class of target data distributions, without the need for assumptions such as smoothness or log-concavity. Our theory is robust vis-a-vis inexact score estimation, degrading gracefully as the score estimation error increases -- without demanding higher-order smoothness on the score estimates as assumed in previous work. The proposed algorithm draws insight from high-order ODE solvers, leveraging high-order Lagrange interpolation and successive refinement to approximate the integral derived from the probability flow ODE. More broadly, our work develops a theoretical framework towards understanding the efficacy of high-order methods for accelerated sampling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。