arXiv:2504.05161stat.MLcs.DS2025-04被引 5

将扩散模型的得分估计与参数和密度估计统一,揭示其统计效率并解决经典难题。

DDPM Score Matching and Distribution Learning

  • 把得分估计转化为参数与密度估计问题,建立理论桥梁。
  • 证明了在多峰分布下,去噪得分匹配具渐近高效性。
  • 首次给出得分估计的计算下界,适用于高维混合高斯模型。

得分估计是基于得分生成模型(SGMs)的核心,尤其在去噪扩散概率模型(DDPMs)中。已有研究显示,准确的得分估计可使SGMs高效生成任意真实数据分布的样本。然而,这种隐式采样器分布的学习结果并未解释得分估计与经典参数和密度估计任务的关系。本文提出一个框架,将得分估计还原为这两类任务,带来统计与计算学习理论的多重启示:参数估计方面,此前工作表明得分匹配对多峰密度的参数估计存在统计低效;本文证明在温和条件下,DDPM中的去噪得分匹配具渐近效率。密度估计方面,通过建立生成与得分估计的联系,我们将现有得分估计保证推广至(ε,δ)-PAC密度估计,即在几乎所有空间中,函数逼近目标对数密度误差小于ε。我们给出了霍尔德类上的极小极大率,并为经典高斯位置混合模型设计出准多项式时间的PAC密度估计算法,解决了Gatmiry等(arXiv'24)提出的开放问题。此外,本框架提供了首个针对一般分布得分估计的计算下界证明方法,应用中建立了广义高斯混合模型的密码学下界,概念上复现了Song(NeurIPS'24)的结果,并推进了其关键开放问题。

原文摘要 · Abstract (English)

Score estimation is the backbone of score-based generative models (SGMs), especially denoising diffusion probabilistic models (DDPMs). A key result in this area shows that with accurate score estimates, SGMs can efficiently generate samples from any realistic data distribution (Chen et al., ICLR'23; Lee et al., ALT'23). This distribution learning result, where the learned distribution is implicitly that of the sampler's output, does not explain how score estimation relates to classical tasks of parameter and density estimation. This paper introduces a framework that reduces score estimation to these two tasks, with various implications for statistical and computational learning theory: Parameter Estimation: Koehler et al. (ICLR'23) demonstrate that a score-matching variant is statistically inefficient for the parametric estimation of multimodal densities common in practice. In contrast, we show that under mild conditions, denoising score-matching in DDPMs is asymptotically efficient. Density Estimation: By linking generation to score estimation, we lift existing score estimation guarantees to $(ε,δ)$-PAC density estimation, i.e., a function approximating the target log-density within $ε$ on all but a $δ$-fraction of the space. We provide (i) minimax rates for density estimation over Hölder classes and (ii) a quasi-polynomial PAC density estimation algorithm for the classical Gaussian location mixture model, building on and addressing an open problem from Gatmiry et al. (arXiv'24). Lower Bounds for Score Estimation: Our framework offers the first principled method to prove computational lower bounds for score estimation across general distributions. As an application, we establish cryptographic lower bounds for score estimation in general Gaussian mixture models, conceptually recovering Song's (NeurIPS'24) result and advancing his key open problem.

生成模型得分匹配统计学习理论分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。