首次给出平滑密度下得分估计的最优统计速率，证明扩散采样可达到理论最优精度。
From optimal score matching to optimal sampling
- 建立光滑紧支撑密度得分估计的极小极大率，揭示理论极限
- 采样分布与真实分布间总变差距离平方期望为 n^{-2α/(2α+1)}
- 无需早停或对数项，适用于高保真生成任务
近年来，基于得分的扩散模型在图像、音频和视频生成中取得显著进展。核心步骤是得分匹配，即从训练数据估计前向扩散过程的得分函数。已有研究表明，生成样本与真实分布之间的总变差距离受得分匹配风险控制。尽管广泛应用，关于得分估计的精确最优统计速率及其在密度估计中的应用仍存在基本理论空白。本文针对光滑、紧支撑密度，建立了得分估计的极小极大率：给定 n 个来自未知 α-霍尔德密度 f（定义于 [-1,1]）的独立同分布样本，估计扩散分布 f * N(0,t) 的得分函数在得分匹配损失下的极小极大率为 (1/(nt²)) ∧ (1/(nt^{3/2})) ∧ (t^{α−1} + n^{-2(α−1)/(2α+1)})，适用于所有 α > 0 及 t ≥ 0。由此可得，由扩散模型生成的样本分布 ̂f 满足期望总变差距离平方 ε(τδ(̂f,f)^2) ≲ n^{-2α/(2α+1)}，且无文献中常见的额外对数项，亦无需早停，这是目前最优结果。
原文摘要 · Abstract (English)
The recent, impressive advances in algorithmic generation of high-fidelity image, audio, and video are largely due to great successes in score-based diffusion models. A key implementing step is score matching, that is, the estimation of the score function of the forward diffusion process from training data. As shown in earlier literature, the total variation distance between the law of a sample generated from the trained diffusion model and the ground truth distribution can be controlled by the score matching risk. Despite the widespread use of score-based diffusion models, basic theoretical questions concerning exact optimal statistical rates for score estimation and its application to density estimation remain open. We establish the sharp minimax rate of score estimation for smooth, compactly supported densities. Formally, given \(n\) i.i.d. samples from an unknown \(α\)-Hölder density \(f\) supported on \([-1, 1]\), we prove the minimax rate of estimating the score function of the diffused distribution \(f * \mathcal{N}(0, t)\) with respect to the score matching loss is \(\frac{1}{nt^2} \wedge \frac{1}{nt^{3/2}} \wedge (t^{α-1} + n^{-2(α-1)/(2α+1)})\) for all \(α> 0\) and \(t \ge 0\). As a consequence, it is shown the law \(\hat{f}\) of a sample generated from the diffusion model achieves the sharp minimax rate \(\bE(\dTV(\hat{f}, f)^2) \lesssim n^{-2α/(2α+1)}\) for all \(α> 0\) without any extraneous logarithmic terms which are prevalent in the literature, and without the need for early stopping which has been required for all existing procedures to the best of our knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。