无需训练数据,通过优化缩放参数实现扩散模型的高精度量化卷积。
Data-Free Group-Wise Fully Quantized Winograd Convolution via Learnable Scales
- 提出仅微调变换矩阵缩放参数的分组量化方法。
- 8比特全量化下生成图像质量接近原始模型(FID与CLIP得分)。
- 适用于无数据场景,适合部署大模型的高效推理需求。
尽管大规模文本到图像扩散模型在复杂视觉任务中取得突破性进展,但其极高的计算与存储成本限制了实际应用。为降低计算开销,已有研究探索对扩散模型进行量化。为进一步提升推理速度,可使用快速卷积算法如Winograd加速卷积层——这是扩散模型中占主导地位的计算部分。然而,现有粗粒度后训练量化方法在完全量化后的Winograd卷积中导致显著质量下降,且对大型模型微调Winograd变换矩阵以恢复性能复杂且代价高昂,因此不适用于大规模基础模型。针对变换矩阵值域范围大的问题,我们研究细粒度分组量化在扩散模型量化中的效果。虽然分组量化能缓解完全量化问题,但在大量Winograd域计算中仍面临分布不平衡挑战。为此,我们仅微调Winograd变换矩阵的缩放参数,无需任何领域特定训练数据。由于方法不依赖训练数据,量化模型的泛化性能得以保障。在文生图任务中,8比特全量化扩散模型采用Winograd实现近乎无损质量(FID和CLIP分数接近全精度模型)。在图像分类任务中,本方法在ResNet18和ResNet-34上分别优于当前最优的Winograd PTQ方法1.62%和2.56%的Top-1 ImageNet准确率,使用Winograd F(6, 3)算子。
原文摘要 · Abstract (English)
Despite the revolutionary breakthroughs of large-scale text-to-image diffusion models for complex vision and downstream tasks, their extremely high computational and storage costs limit their usability. Quantization of diffusion models has been explored in recent works to reduce compute costs and memory bandwidth usage. To further improve inference time, fast convolution algorithms such as Winograd can be used for convolution layers, which account for a significant portion of computations in diffusion models. However, the significant quality loss of fully quantized Winograd using existing coarser-grained post-training quantization methods, combined with the complexity and cost of finetuning the Winograd transformation matrices for such large models to recover quality, makes them unsuitable for large-scale foundation models. Motivated by the presence of a large range of values in them, we investigate the impact of finer-grained group-wise quantization in quantizing diffusion models. While group-wise quantization can largely handle the fully quantized Winograd convolution, it struggles to deal with the large distribution imbalance in a sizable portion of the Winograd domain computation. To reduce range differences in the Winograd domain, we propose finetuning only the scale parameters of the Winograd transform matrices without using any domain-specific training data. Because our method does not depend on any training data, the generalization performance of quantized diffusion models is safely guaranteed. For text-to-image generation task, the 8-bit fully-quantized diffusion model with Winograd provides near-lossless quality (FID and CLIP scores) in comparison to the full-precision model. For image classification, our method outperforms the state-of-the-art Winograd PTQ method by 1.62% and 2.56% in top-1 ImageNet accuracy on ResNet18 and ResNet-34, respectively, with Winograd F(6, 3).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。