用向量量化压缩扩散模型，极低比特下仍保持高质量生成。
AQUATIC-Diff: Additive Quantization for Truly Tiny Compressed Diffusion Models
- 采用基于码本的加性向量量化，对权重组进行整体压缩。
- 在W2A8下实现最低sFID（1.92点更低）和最优FID、ISC指标。
- 支持任意硬件的高效推理，避免依赖特定小整数指令集。
尽管扩散模型在多模态生成中广泛应用，但其推理对硬件资源要求极高。现有量化方法多采用均匀标量量化（USQ），而向量量化（VQ）在大语言模型中表现优异。本文将基于码本的加性向量量化应用于扩散模型压缩，在标准类条件基准LDM-4 on ImageNet上，于20次推理步数下达到新的帕累托前沿。结果显示，W4A8时sFID比全精度模型低1.92点，且在W2A8下首次取得最佳的FID、sFID和ISC性能。此外，通过高效推理内核实现任意硬件上的计算量减少，而非依赖仅部分设备支持的小整数运算。
原文摘要 · Abstract (English)
Significant investments have been made towards the commodification of diffusion models for generation of diverse media. Their mass-market adoption is however still hobbled by the intense hardware resource requirements of diffusion model inference. Model quantization strategies tailored specifically towards diffusion models have been useful in easing this burden, yet have generally explored the Uniform Scalar Quantization (USQ) family of quantization methods. In contrast, Vector Quantization (VQ) methods, which operate on groups of multiple related weights as the basic unit of compression, have seen substantial success in Large Language Model (LLM) quantization. In this work, we apply codebook-based additive vector quantization to the problem of diffusion model compression. Our resulting approach achieves a new Pareto frontier for the extremely low-bit weight quantization on the standard class-conditional benchmark of LDM-4 on ImageNet at 20 inference time steps. Notably, we report sFID 1.92 points lower than the full-precision model at W4A8 and the best-reported results for FID, sFID and ISC at W2A8. We are also able to demonstrate FLOPs savings on arbitrary hardware via an efficient inference kernel, as opposed to savings resulting from small integer operations which may lack broad hardware support.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。