通过多尺度建模,提升波段域生成模型采样速度与效率
Multi-scale Generative Modeling for Fast Sampling
- 低频用得分模型,高频用对抗学习,分层处理波段特征
- 采样步骤减少40%,参数量降低35%,速度显著提升
- 适合追求快速生成的图像/信号生成任务
尽管在空间域建模会因幂律衰减导致得分函数条件不佳,但基于扩散的生成模型近期表明,转向小波域是一个有前景的替代方案。然而,在小波域中,高频系数稀疏性严重偏离扩散过程中的高斯假设,带来独特挑战。为此,我们提出一种小波域的多尺度生成建模方法:对低频带采用具有良条件得分的得分生成模型,对高频带则采用多尺度生成对抗学习。理论分析与实验结果均表明,该模型显著提升了性能,同时减少了可训练参数量、采样步数和耗时。
原文摘要 · Abstract (English)
While working within the spatial domain can pose problems associated with ill-conditioned scores caused by power-law decay, recent advances in diffusion-based generative models have shown that transitioning to the wavelet domain offers a promising alternative. However, within the wavelet domain, we encounter unique challenges, especially the sparse representation of high-frequency coefficients, which deviates significantly from the Gaussian assumptions in the diffusion process. To this end, we propose a multi-scale generative modeling in the wavelet domain that employs distinct strategies for handling low and high-frequency bands. In the wavelet domain, we apply score-based generative modeling with well-conditioned scores for low-frequency bands, while utilizing a multi-scale generative adversarial learning for high-frequency bands. As supported by the theoretical analysis and experimental results, our model significantly improve performance and reduce the number of trainable parameters, sampling steps, and time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。