首次证明扩散模型的确定性微分方程采样器在总变差距离下近似最优。
Minimax Optimality of the Probability Flow ODE for Diffusion Models
- 提出平滑正则化得分估计器,同时控制得分误差与雅可比误差。
- 在亚高斯分布上实现总变差距离的极小极大最优率(对数因子内)。
- 无需密度下界或光滑性假设,适用于广泛实际数据分布。
基于得分的扩散模型已成为现代生成建模的基础范式,能从复杂高维分布中生成样本。尽管概率流常微分方程(ODE)采样器因采样效率和精度高而被广泛采用,但其严格的统计保证在文献中长期缺失。本文首次构建了确定性ODE采样器的完整理论框架,在目标分布为β-霍尔德光滑密度的亚高斯分布(β ≤ 2)这一温和假设下,提出一种平滑正则化得分估计器,同时控制$L^2$得分误差与对应的均值雅可比误差。通过改进的ODE采样过程收敛性分析,证明所得采样器在总变差距离上达到极小极大率(对数因子内)。该理论全面考虑采样过程中的所有误差来源,且无需密度下界或目标分布得分的利普希茨/光滑性等强结构条件,覆盖了广泛的实用数据分布。
原文摘要 · Abstract (English)
Score-based diffusion models have become a foundational paradigm for modern generative modeling, demonstrating exceptional capability in generating samples from complex high-dimensional distributions. Despite the dominant adoption of probability flow ODE-based samplers in practice due to their superior sampling efficiency and precision, rigorous statistical guarantees for these methods have remained elusive in the literature. This work develops the first end-to-end theoretical framework for deterministic ODE-based samplers that establishes near-minimax optimal guarantees under mild assumptions on target data distributions. Specifically, focusing on subgaussian distributions with $β$-Hölder smooth densities for $β\leq 2$, we propose a smooth regularized score estimator that simultaneously controls both the $L^2$ score error and the associated mean Jacobian error. Leveraging this estimator within a refined convergence analysis of the ODE-based sampling process, we demonstrate that the resulting sampler achieves the minimax rate in total variation distance, modulo logarithmic factors. Notably, our theory comprehensively accounts for all sources of error in the sampling process and does not require strong structural conditions such as density lower bounds or Lipschitz/smooth scores on target distributions, thereby covering a broad range of practical data distributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。