提出新方法，让生成模型更快更准地估计数据密度梯度。
Implicit score matching meets denoising score matching: improved rates of convergence and log-density Hessian estimation
- 结合隐式与去噪得分匹配，利用低维结构提升效率。
- 样本量相同时，收敛速度与去噪法相当，达到最优率。
- 无需高维计算即可估计密度黑塞矩阵，适合生成模型应用。
我们研究了在数据分布具有低维结构的前提下，利用隐式得分匹配和去噪得分匹配估计得分函数的问题。证明隐式得分匹配不仅能适应内在维度，还能在样本规模上达到与去噪得分匹配相同的收敛速率。此外，两种方法均可通过简单微分实现对对数密度黑塞矩阵的估计，且不受维数诅咒影响，从而为基于常微分方程的采样器提供收敛性保证。该方法依赖于类Gagliardo-Nirenberg不等式，关联加权$L^2$范数与其导数的尺度关系。
原文摘要 · Abstract (English)
We study the problem of estimating the score function using both implicit score matching and denoising score matching. Assuming that the data distribution exhibiting a low-dimensional structure, we prove that implicit score matching is able not only to adapt to the intrinsic dimension, but also to achieve the same rates of convergence as denoising score matching in terms of the sample size. Furthermore, we demonstrate that both methods allow us to estimate log-density Hessians without the curse of dimensionality by simple differentiation. This justifies convergence of ODE-based samplers for generative diffusion models. Our approach is based on Gagliardo-Nirenberg-type inequalities relating weighted $L^2$-norms of smooth functions and their derivatives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。