通过似然匹配训练扩散模型,提升生成质量与收敛性。
Likelihood Matching for Diffusion Models
- 用高斯近似反向扩散每步密度,匹配均值与协方差。
- 同时估计得分与海森矩阵,实现前后两阶矩一致匹配。
- 理论保证收敛速率,适合需高精度生成的研究者。
我们提出一种似然匹配方法来训练扩散模型,首先建立目标数据分布的似然与反向扩散样本路径上的似然之间的等价关系。为高效计算反向样本似然,采用拟似然近似每一步反向转移密度,将其近似为均值和协方差匹配的高斯分布。通过最大化拟似然来估计扩散生成中的得分与海森函数,确保任意两个时间点间一阶与二阶转移矩的一致匹配。引入随机采样器以利用估计的得分与海森信息。我们建立了拟最大似然估计的一致性,并为所提采样器提供了非渐近收敛保证,量化了因得分与海森估计、维度及扩散步数带来的近似误差率。实验与模拟评估验证了所提似然匹配的有效性,并支持理论结果。
原文摘要 · Abstract (English)
We propose a Likelihood Matching approach for training diffusion models by first establishing an equivalence between the likelihood of the target data distribution and a likelihood along the sample path of the reverse diffusion. To efficiently compute the reverse sample likelihood, a quasi-likelihood is considered to approximate each reverse transition density by a Gaussian distribution with matched conditional mean and covariance, respectively. The score and Hessian functions for the diffusion generation are estimated by maximizing the quasi-likelihood, ensuring a consistent matching of both the first two transitional moments between every two time points. A stochastic sampler is introduced to facilitate computation that leverages both the estimated score and Hessian information. We establish consistency of the quasi-maximum likelihood estimation, and provide non-asymptotic convergence guarantees for the proposed sampler, quantifying the rates of the approximation errors due to the score and Hessian estimation, dimensionality, and the number of diffusion steps. Empirical and simulation evaluations demonstrate the effectiveness of the proposed Likelihood Matching and validate the theoretical results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。