arXiv:2607.04442stat.MLcs.LG2026-07

改进扩散模型的分数匹配间隙,提升生成质量理论上限

Tightening the Score Matching Gap for Diffusion Models

论文配图:Tightening the Score Matching Gap for Diffusion Models
图 1 · 摘自论文原文
  • 基于后向过程收缩性与熵流分析,推导更紧的分数匹配上界
  • 在低噪声尺度下,分数估计精度对缩小间隙起决定作用
  • 适用于关注生成质量理论保障的研究者与模型优化工程师

扩散模型(DMs)是当前最先进的生成方法,通过近似采样未知分布实现生成。其训练与评估主要依赖证据下界(ELBO),该下界将模型样本的KL散度与路径上的分数匹配损失关联,后者作为可计算的代理目标。然而,样本质量与分数匹配损失之间的差异构成所谓的“分数匹配间隙”,虽在最坏情况下为紧致但通常无法准确反映实际生成质量。本文提供对该间隙的理论分析,针对三种度量——KL散度、反向KL散度和Wasserstein距离——建立了更紧的上界,有效利用了分数估计器类别的正则性。关键技术洞察在于利用后向过程的收缩性质,结合熵流、对数索博列夫不等式与反射耦合,严格连接朗之万扩散的遍历性与分数匹配间隙问题。结果表明,在低噪声尺度下,分数近似的质量对缩小间隙具有决定性影响。

原文摘要 · Abstract (English)

Diffusion models (DMs) are a state-of-the-art generative method to approximately sample from an unknown distribution. Their training and evaluation primarily rely on an Evidence Lower Bound (ELBO), which relates the Kullback-Leibler (KL) divergence of model samples to the score matching loss along the path, which serves as a tractable surrogate. The difference between sample quality and the score matching loss produced by this bound leads to the \emph{score matching gap}, which is known to be tight in the worst-case but not descriptive of sample quality in general. In this work, we provide a theoretical analysis of this gap, developing tighter bounds for three metrics: KL divergence, reverse KL divergence, and Wasserstein distance, effectively exploiting the regularity of the class of score estimators. Our results suggest that the quality of the score approximation has more impact on closing the score matching gap for low noise scales. To obtain these bounds, our key technical insight is to exploit the contraction properties of the backward processes. In particular, we rely on entropy flows, logarithmic Sobolev inequalities and reflection couplings, rigorously linking the ergodicity of the Langevin diffusion to the score matching gap problem.

扩散模型分数匹配生成模型理论分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。