arXiv:2412.06661cs.CV2024-12被引 5

提出新量化框架,让扩散模型在提速同时保持图像质量一致。

Efficiency Meets Fidelity: A Novel Quantization Framework for Stable Diffusion

  • 设计串行到并行的量化流水线,保证量化前后生成结果一致。
  • 在W4A8下显著提升图像质量和分布相似性,优于现有方法。
  • 适合追求高效与高保真图像生成的工业级应用开发人员。

通过稳定扩散模型(SDM)进行文本到图像生成展现出强大能力,但其计算密集的迭代去噪过程阻碍了对延迟敏感场景的实时部署。尽管已有研究探索后训练量化(PTQ)和量化感知训练(QAT)以压缩扩散模型,但现有方法常忽视量化模型与浮点模型生成结果的一致性。这种一致性在专业应用场景中至关重要,需兼顾效率与输出可靠性。为此,我们提出一种高效的稳定扩散模型量化框架。该框架引入串行到并行的流水线,同时保障训练-推理一致性与优化稳定性。在此基础上,进一步提出多时间步激活量化、时间信息预计算、层间知识蒸馏及选择性冻结等技术,在保持量化效率的同时,实现与浮点模型相当的高质量生成效果。在多个稳定扩散变体(v1-4、v2-1、XL 1.0 和 v3)上进行全面评估,本方法在训练时间更短的前提下,相比当前最优方法表现更优。在 W4A8 量化设置下,显著提升了分布相似性和视觉保真度,同时维持高图像质量。

原文摘要 · Abstract (English)

Text-to-image generation via Stable Diffusion models (SDM) have demonstrated remarkable capabilities. However, their computational intensity, particularly in the iterative denoising process, hinders real-time deployment in latency-sensitive applications. While Recent studies have explored post-training quantization (PTQ) and quantization-aware training (QAT) methods to compress Diffusion models, existing methods often overlook the consistency between results generated by quantized models and those from floating-point models. This consistency is paramount for professional applications where both efficiency and output reliability are essential. To ensure that quantized SDM generates high-quality and consistent images, we propose an efficient quantization framework for SDM. Our framework introduces a Serial-to-Parallel pipeline that simultaneously maintains training-inference consistency and ensures optimization stability. Building upon this foundation, we further develop several techniques including multi-timestep activation quantization, time information precalculation, inter-layer distillation, and selective freezing, to achieve high-fidelity generation in comparison to floating-point models while maintaining quantization efficiency. Through comprehensive evaluation across multiple Stable Diffusion variants (v1-4, v2-1, XL 1.0, and v3), our method demonstrates superior performance over state-of-the-art approaches with shorter training times. Under W4A8 quantization settings, we achieve significant improvements in both distribution similarity and visual fidelity, while preserving a high image quality.

扩散模型量化图像生成高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。