优化高维生成模型噪声调度，提升采样精度与效率。
Optimizing Noise Schedules of Generative Models in High Dimensionss
- 基于随机插值框架，分析不同噪声调度对特征恢复的影响。
- 特定噪声调度使模型在高维下仅需Θ_d(1)步即可完成采样，优于常数去噪的Θ_d(√d)步。
- 适用于需要高效高维生成的场景，如复杂分布建模与扩散模型优化。
近期研究发现扩散模型存在相变现象，准确生成样本需合理设计噪声调度。本文在随机插值框架下重新审视常见的方差保持（VP）与方差爆炸（VE）调度。以高斯混合（GM）和居里-韦斯（CW）数据分布为测试模型，结果表明：当每步去噪量恒定时，VP能恢复低层特征（各模式分布），但忽略高层特征（模式间不对称性）；反之，VE表现相反。通过设计针对VP与VE的特定噪声调度，可同时恢复高低层特征。最终，该方法使生成模型的概率流ODE在维度d下仅需Θ_d(1)步离散化，远优于常数去噪所需的Θ_d(√d)步。
原文摘要 · Abstract (English)
Recent works have shown that diffusion models can undergo phase transitions, the resolution of which is needed for accurately generating samples. This has motivated the use of different noise schedules, the two most common choices being referred to as variance preserving (VP) and variance exploding (VE). Here we revisit these schedules within the framework of stochastic interpolants. Using the Gaussian Mixture (GM) and Curie-Weiss (CW) data distributions as test case models, we first investigate the effect of the variance of the initial noise distribution and show that VP recovers the low-level feature (the distribution of each mode) but misses the high-level feature (the asymmetry between modes), whereas VE performs oppositely. We also show that this dichotomy, which happens when denoising by a constant amount in each step, can be avoided by using noise schedules specific to VP and VE that allow for the recovery of both high- and low-level features. Finally we show that these schedules yield generative models for the GM and CW model whose probability flow ODE can be discretized using $Θ_d(1)$ steps in dimension $d$ instead of the $Θ_d(\sqrt{d})$ steps required by constant denoising.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。