提出StAD方法，无需计算雅可比即可快速估算扩散模型的似然。
StAD: Stein Amortized Divergence for Fast Likelihoods with Diffusion and Flow
- 用Langevin-Stein算子蒸馏预测概率流的散度，避免计算雅可比。
- 在CIFAR-10和ImageNet上比Hutchinson方法方差更小、速度更快。
- 适用于多种生成模型，适合需要高效似然估计的研究者。
扩散模型和基于流的模型广泛用于生成建模与密度估计。它们具有确定性的概率流常微分方程（PF-ODE），类似于连续归一化流（CNFs），描述概率质量的迁移过程。获取这些模型的似然对许多工作流程（尤其是贝叶斯分析）至关重要，但需计算雅可比矩阵的迹以得到学习到的PF-ODE的散度，这要么精确计算需$Ø(D^2)$复杂度，要么使用噪声估计仅需$Ø(D)$。我们提出StAD，一种新蒸馏方法，利用Langevin-Stein算子预测并学习PF-ODE的散度，无需任何雅可比计算。实验表明，该方法在CIFAR-10、ImageNet等密度估计任务上表现优于Hutchinson和Hutch++，显著降低似然预测的方差并提升速度。此外，我们证明该方法可推广至多样化的生成模型，并在某些正则条件下使学习到的向量场满足Stein类要求。
原文摘要 · Abstract (English)
Diffusion and flow-based models are ubiquitously used for generative modelling and density estimation. They admit a deterministic probability flow ordinary differential equation (PF-ODE), analogous to continuous normalizing flows (CNFs), which describes the transport of the probability mass. Obtaining the likelihood from these models is of interest to many workflows, especially Bayesian analysis, and requires solving the trace of the Jacobian to compute the divergence of the learned PF-ODE, which is either $\mathcal{O}(D^2)$ to compute exactly or $\mathcal{O}(D)$ with a noisy estimate. We introduce StAD, a new distillation method to predict and learn the divergence of the PF-ODE using the Langevin-Stein operator without ever computing the Jacobian. We show that our method is competitive with the Hutchinson and Hutch++ on CIFAR-10, ImageNet and other density estimation tasks, consistently improving the variance and speed of the likelihood predictions compared to the Hutchinson. We additionally show our method will generalize to a varied class of generative models, and show that under some regularity conditions these learned vector fields can be made to satisfy the Stein class.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。