arXiv:2606.06179stat.MLcs.LG2026-06被引 2

发现扩散模型的梯度误差才是关键,纯数学误差可能骗人。

Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors

  • 用几何分解法分离出可观察的梯度误差和不可见的漩涡误差
  • 证明了传统误差指标无法有效衡量生成分布质量
  • 提出新误差估计器,与真实生成质量相关性显著提升

基于分数的扩散模型通常通过最小化 $L^2$ 分数匹配误差进行训练,现有理论分析也依赖该量来控制生成分布与目标分布之间的差异。我们发现 $L^2$ 分数误差并非衡量边缘分布质量的内在指标:学习到的扩散模型可能具有任意大的 $L^2$ 分数误差,却仍能完全匹配目标分布。通过将分数误差分解为梯度分量与无散分量(赫尔姆霍兹-霍奇分解),我们揭示其几何根源:只有梯度分量影响边缘福克-普朗克动力学,而无散分量在结构上不可见。我们给出三个精确结果:第一,基于修正的几何视角,证明任何单调函数的 $L^2$ 分数误差都无法统一下界任意分布差异;第二,推导出仅依赖可观测梯度分量的互信息(KL)上界,收紧了通用分数网络的标准吉尔萨诺夫界,并指出其松散性源于路径空间而非边缘空间的动力学建模代价;第三,通过双重索博列夫恒等式提出一种可计算的梯度分量估计器,实验证明其与样本质量的相关性远高于全量 $L^2$ 误差。

原文摘要 · Abstract (English)

Score-based diffusion models are typically trained by minimizing the $L^2$ score matching error, and standard theoretical analyses rely on this quantity to bound the sampling discrepancy between the learned and target distributions. We show the $L^2$ score error is not the right intrinsic measure of marginal distributional quality: a learned diffusion model can incur arbitrarily large $L^2$ score error while perfectly matching the target distribution. By decomposing score errors into a gradient and a solenoidal component (a Helmholtz-Hodge decomposition), we identify the geometric reason behind this: only the gradient component enters the marginal Fokker-Planck dynamics, while the solenoidal component is structurally invisible. We make this precise in three results. First, building on the corrected geometry, we prove an impossibility result: no monotone function of the $L^2$ score error can uniformly lower bound any divergence between the learned and target distributions. Second, we derive an upper bound on the Kullback-Leibler divergence that depends only on the observable gradient component of the error, tightening the standard Girsanov bound for generic score networks, and identifying its looseness as the cost of operating on path-space rather than marginal-space dynamics. Third, we give a tractable estimator of the gradient component via a dual Sobolev identity, which is shown to empirically correlate substantially better with sample quality than the full $L^2$ error.

扩散模型分数匹配几何分析误差评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。