arXiv:2410.23285cs.LGcs.AI2024-10被引 29

提出无需训练的加速采样方法,显著提升扩散模型采样速度。

Provable Acceleration for Diffusion Models under Minimal Assumptions

  • 不依赖训练,基于最小假设实现采样加速
  • 在ε≤1/√d条件下,迭代次数降至d^{5/4}/√ε量级
  • 理论严谨,适用于广泛分布类型,适合研究者参考

基于得分的扩散模型虽在采样上达到极小极大最优,但因得分函数计算开销大,采样速度慢。尽管近期已有大量加速采样的实证进展,其理论理解仍十分有限。本文提出一种无需训练的随机采样加速方案。在仅需$L^2$精度得分估计及目标分布二阶矩有界的最简假设下,所提加速采样器在总变差距离上达到$\ ilde{O}(d^{5/4}/\sqrt{\varepsilon})$次迭代即获$\\ ext{\varepsilon}$-精度,显著优于标准得分模型的$\\widetilde{O}(d/\varepsilon)$复杂度(当$\\ ext{\varepsilon} \leq 1/\sqrt{d}$时)。该理论不依赖对目标分布的强假设或高阶得分估计保证。

原文摘要 · Abstract (English)

Score-based diffusion models, while achieving minimax optimality for sampling, are often hampered by slow sampling speeds due to the high computational burden of score function evaluations. Despite the recent remarkable empirical advances in speeding up the score-based samplers, theoretical understanding of acceleration techniques remains largely limited. To bridge this gap, we propose a novel training-free acceleration scheme for stochastic samplers. Under minimal assumptions -- namely, $L^2$-accurate score estimates and a finite second-moment condition on the target distribution -- our accelerated sampler provably achieves $\varepsilon$-accuracy in total variation within $\widetilde{O}(d^{5/4}/\sqrt{\varepsilon})$ iterations, thereby significantly improving upon the $\widetilde{O}(d/\varepsilon)$ iteration complexity of standard score-based samplers for $\varepsilon\leq 1/\sqrt{d}$. Notably, our convergence theory does not rely on restrictive assumptions on the target distribution or higher-order score estimation guarantees.

扩散模型采样加速理论分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。