提出双阶段去噪机制,让量化后的扩散模型生成更清晰图像。
D$^2$-DPM: Dual Denoising for Quantized Diffusion Probabilistic Models
- 分离量化噪声对均值和方差的影响,分步去噪提升精度。
- 在3.99倍压缩下,图像质量比全精度模型高1.42个FID。
- 适合低延迟、资源受限场景的高效图像生成应用。
扩散模型在图像生成中表现卓越,但其漫长的去噪过程和计算密集的得分估计网络限制了在低延迟和资源受限场景下的可扩展性。后训练量化(PTQ)可在不重新训练的情况下压缩并加速扩散模型,但会引入额外的量化噪声,导致均值和方差偏移。本文提出D2-DPM,一种双阶段去噪机制,旨在精确缓解量化噪声对得分估计网络的负面影响。具体地,我们解析量化噪声对采样方程的影响为均值偏差和方差偏差:前者改变采样轨迹的趋势,后者放大扩散系数,影响收敛性。所提D2-DPM在每个时间步先去除量化噪声,再通过逆向扩散迭代净化样本。实验表明,D2-DPM在实现3.99倍压缩和11.67倍比特操作加速的同时,生成质量优于全精度模型,FID降低1.42。
原文摘要 · Abstract (English)
Diffusion models have achieved cutting-edge performance in image generation. However, their lengthy denoising process and computationally intensive score estimation network impede their scalability in low-latency and resource-constrained scenarios. Post-training quantization (PTQ) compresses and accelerates diffusion models without retraining, but it inevitably introduces additional quantization noise, resulting in mean and variance deviations. In this work, we propose D2-DPM, a dual denoising mechanism aimed at precisely mitigating the adverse effects of quantization noise on the noise estimation network. Specifically, we first unravel the impact of quantization noise on the sampling equation into two components: the mean deviation and the variance deviation. The mean deviation alters the drift coefficient of the sampling equation, influencing the trajectory trend, while the variance deviation magnifies the diffusion coefficient, impacting the convergence of the sampling trajectory. The proposed D2-DPM is thus devised to denoise the quantization noise at each time step, and then denoise the noisy sample through the inverse diffusion iterations. Experimental results demonstrate that D2-DPM achieves superior generation quality, yielding a 1.42 lower FID than the full-precision model while achieving 3.99x compression and 11.67x bit-operation acceleration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。