arXiv:2411.01629stat.MLcs.LG2024-11被引 6

揭示扩散模型去噪的难易本质:曲率复杂度决定去噪精度

Denoising Diffusions with Optimal Transport: Localization, Curvature, and Multi-Scale Complexity

  • 从最优传输视角看去噪,提出曲率决定定位不确定性
  • 发现多尺度曲率复杂度决定去噪难度,非最坏情况而是平均表现
  • 适用于理解非对数凹分布生成中的瓶颈信噪比

基于扩散的生成模型旨在将一条朗之万扩散链从一个对数凹的平衡测度ν(如各向同性高斯)反向去噪至复杂的初始测度μ。得分函数执行去噪,逆时间方向移动,并预测给定当前状态时过去位置的条件均值。本文证明得分去噪是最优传输成本下的反向映射。我们发现曲率函数决定了这种定位不确定性的程度,以给定当前状态下过去位置的条件方差衡量。研究扩散-去噪过程的有效性:前向扩散的收缩被后向去噪链的可能扩张抵消,共同决定去噪难度。对于任意初始测度μ,我们证明该时间t处的净收缩由在特定信噪比(SNR)尺度r(t)下平滑μ的曲率复杂度刻画。我们发现多尺度曲率复杂度共同决定去噪链的难度。该复杂度量化了一种精细的平均曲率概念而非最坏情况。有趣的是,它依赖于一个积分尾函数,测量正曲率区域与负曲率区域相对质量;若该尾部较轻,则特定信噪比下的去噪较容易。最后通过多个非对数凹例子展示多尺度复杂度如何探测扩散-去噪过程的瓶颈信噪比。

原文摘要 · Abstract (English)

Adding noise is easy; what about denoising? Diffusion is easy; what about reverting a diffusion? Diffusion-based generative models aim to denoise a Langevin diffusion chain, moving from a log-concave equilibrium measure $ν$, say an isotropic Gaussian, back to a complex, possibly non-log-concave initial measure $μ$. The score function performs denoising, moving backward in time, and predicting the conditional mean of the past location given the current one. We show that score denoising is the optimal backward map in transportation cost. What is its localization uncertainty? We show that the curvature function determines this localization uncertainty, measured as the conditional variance of the past location given the current. We study in this paper the effectiveness of the diffuse-then-denoise process: the contraction of the forward diffusion chain, offset by the possible expansion of the backward denoising chain, governs the denoising difficulty. For any initial measure $μ$, we prove that this offset net contraction at time $t$ is characterized by the curvature complexity of a smoothed $μ$ at a specific signal-to-noise ratio (SNR) scale $r(t)$. We discover that the multi-scale curvature complexity collectively determines the difficulty of the denoising chain. Our multi-scale complexity quantifies a fine-grained notion of average-case curvature instead of the worst-case. Curiously, it depends on an integrated tail function, measuring the relative mass of locations with positive curvature versus those with negative curvature; denoising at a specific SNR scale is easy if such an integrated tail is light. We conclude with several non-log-concave examples to demonstrate how the multi-scale complexity probes the bottleneck SNR for the diffuse-then-denoise process.

扩散模型去噪机制曲率复杂度最优传输

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。