arXiv:2506.11378cs.LG2025-06被引 2

分析随机性对扩散模型采样的影响,揭示其优劣机制。

The Effect of Stochasticity in Score-Based Diffusion Sampling: a KL Divergence Analysis

  • 通过KL散度分析随机性在采样中的作用
  • 随机性可减少误差,但也会放大当前错误
  • 适合研究扩散模型原理与优化的学者

基于得分的扩散模型采样可通过求解反向随机微分方程(SDE)或概率流常微分方程(ODE)实现,前者允许任意随机性函数,后者对应将随机性设为零。本文通过分析KL散度演化,获得一般性边界,并提出新方法研究得分误差的时间分布对性能的影响。对于精确得分函数,随机性具有压缩效应,降低采样路径上的KL散度;而对于近似得分函数,需在纠正累积误差与放大当前误差之间权衡,导致随机性可能提升或降低生成质量。理论表明,随机性的收益取决于训练误差的时间局部性。实验在玩具数据和基准数据集上验证了该结论,并将实际KL散度演化与理论边界对比。此外,还提供一个完全解析的例子,所有量可显式计算,最优随机性函数可通过最优控制分析确定。

原文摘要 · Abstract (English)

Sampling in score-based diffusion models can be performed by solving either a reverse-time stochastic differential equation (SDE) parameterized by an arbitrary stochasticity function or a probability flow ODE, corresponding to setting this stochasticity function to zero. In this work, we investigate the effect of this stochasticity on the generation process through the evolution of Kullback-Leibler (KL) divergences, obtaining general KL divergence bounds and a novel analysis of the impact of the time-profile of the score error on model performance. For exact score functions, stochasticity has a contractive effect, decreasing KL divergence along the sampling trajectory. For approximate scores, however, a trade-off arises between correcting accumulated errors and amplifying current score errors, meaning stochasticity can either improve or degrade generation performance. Theoretical considerations indicate that the gain from stochasticity depends on the time-localization of the trained model error. We test this in experiments on both toy and benchmark data sets, also comparing the KL divergence evolution with the obtained bounds. We also present a fully analytical example, where all the relevant quantities can be computed, and the optimal stochasticity function can be characterized via an optimal control analysis.

扩散模型随机性分析KL散度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。