解释乘性噪声条件下的扩散模型为何有效
Deterministic Dynamics of Sampling Processes in Score-Based Diffusion Models with Multiplicative Noise Conditioning
- 通过研究微分方程的确定性动力学分析模型机制
- 发现即使无法完全学习真实梯度，仍能生成高质量样本
- 为实际表现与理论局限之间的矛盾提供解释，适合研究生成模型原理者
基于得分的扩散模型通过学习扩散过程相关的得分函数来生成新样本。尽管其采样过程可用微分方程理论解释，但Song与Ermon（2020）指出，使用乘性噪声条件的神经网络仍可生成满意样本。在此设置中，模型被表示为依赖空间变量的函数与依赖噪声幅度的函数的乘积，该结构限制了空间变量与噪声之间更一般关系的建模能力，意味着模型无法完全学习正确的得分。然而，实践中这些模型表现良好。本文通过研究相关微分方程的确定性动力学，为这一现象提供了理论解释，揭示了模型的实际运作机制。
原文摘要 · Abstract (English)
Score-based diffusion models generate new samples by learning the score function associated with a diffusion process. While the effectiveness of these models can be theoretically explained using differential equations related to the sampling process, previous work by Song and Ermon (2020) demonstrated that neural networks using multiplicative noise conditioning can still generate satisfactory samples. In this setup, the model is expressed as the product of two functions: one depending on the spatial variable and the other on the noise magnitude. This structure limits the model's ability to represent a more general relationship between the spatial variable and the noise, indicating that it cannot fully learn the correct score. Despite this limitation, the models perform well in practice. In this work, we provide a theoretical explanation for this phenomenon by studying the deterministic dynamics of the associated differential equations, offering insight into how the model operates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。