用文本提示指导扩散模型量化，提升效率与图像质量
Text Embedding Knows How to Quantize Text-Guided Diffusion Models
- 根据文本提示动态调整每层每步的精度
- 在多个数据集上降低计算量并提升生成质量
- 可无缝接入现有量化方法，适合部署优化
尽管扩散模型在文生图等任务中表现优异，但其巨大的计算开销限制了在资源受限环境中的应用。为此，网络量化成为设计高效扩散模型的有前景方案。然而，现有量化方法未将输入条件（如文本提示）视为量化的重要信息源。本文提出一种新量化方法QLIP，利用文本提示指导每层在每个时间步的比特精度选择。QLIP可无缝集成到现有量化方法中，显著提升量化效率。大量实验表明，QLIP在多个数据集上有效降低计算复杂度，并改善生成图像质量。
原文摘要 · Abstract (English)
Despite the success of diffusion models in image generation tasks such as text-to-image, the enormous computational complexity of diffusion models limits their use in resource-constrained environments. To address this, network quantization has emerged as a promising solution for designing efficient diffusion models. However, existing diffusion model quantization methods do not consider input conditions, such as text prompts, as an essential source of information for quantization. In this paper, we propose a novel quantization method dubbed Quantization of Language-to-Image diffusion models using text Prompts (QLIP). QLIP leverages text prompts to guide the selection of bit precision for every layer at each time step. In addition, QLIP can be seamlessly integrated into existing quantization methods to enhance quantization efficiency. Our extensive experiments demonstrate the effectiveness of QLIP in reducing computational complexity and improving the quality of the generated images across various datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。