首次给出分数不匹配扩散模型的理论保证,揭示采样偏差来源。
Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers
- 建立分数不匹配扩散采样的性能上界,明确维度依赖关系。
- 发现目标分布与训练分布的累积偏差导致渐近分布偏移。
- 提出最优偏差零样本采样器,适用于线性条件模型。
去噪扩散模型近年来成为强大的生成技术,可将噪声转化为有意义数据。尽管当目标分布与训练分布一致时,扩散模型的收敛性已有充分理论保障,但实际场景中常存在分布不匹配。一个典型情况是零样本条件采样,此时目标条件分布不同于(无条件)训练分布。这类分数不匹配扩散模型在理论上仍缺乏研究。本文首次为一般分数不匹配扩散采样器提供了带显式维度依赖的性能保证,聚焦于具有有限二阶矩的目标分布。我们证明,分数不匹配会导致目标分布与采样分布之间的渐近分布偏移,其大小与目标分布和训练分布间的累积偏差成正比。该结果可直接应用于任意条件模型的零样本条件采样器,无论是否存在测量噪声。有趣的是,推导出的收敛上界为设计新型偏差最优零样本采样器提供了指导,该采样器在线性条件模型中能最小化渐近偏差。对于此类最优采样器,我们进一步建立了显式依赖维度与条件信息的收敛保证,适用于具有有界支撑和高斯混合等有趣目标分布。研究结果得到数值实验支持。
原文摘要 · Abstract (English)
The denoising diffusion model has recently emerged as a powerful generative technique, capable of transforming noise into meaningful data. While theoretical convergence guarantees for diffusion models are well established when the target distribution aligns with the training distribution, practical scenarios often present mismatches. One common case is in the zero-shot conditional diffusion sampling, where the target conditional distribution is different from the (unconditional) training distribution. These score-mismatched diffusion models remain largely unexplored from a theoretical perspective. In this paper, we present the first performance guarantee with explicit dimensional dependencies for general score-mismatched diffusion samplers, focusing on target distributions with finite second moments. We show that score mismatches result in an asymptotic distributional bias between the target and sampling distributions, proportional to the accumulated mismatch between the target and training distributions. This result can be directly applied to zero-shot conditional samplers for any conditional model, irrespective of measurement noise. Interestingly, the derived convergence upper bound offers useful guidance for designing a novel bias-optimal zero-shot sampler in linear conditional models that minimizes the asymptotic bias. For such bias-optimal samplers, we further establish convergence guarantees with explicit dependencies on dimension and conditioning, applied to several interesting target distributions, including those with bounded support and Gaussian mixtures. Our findings are supported by numerical studies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。