用分数比特量化让大模型在手机上高效运行且保持生成质量
FraQAT: Quantization Aware Training with Fractional bits
- 通过逐步降低精度并引入分数比特优化，提升量化后模型质量
- 在多个扩散模型上实现4%-7%更低的FiD指标，优于标准QAT
- 适合需要在移动设备部署高质量生成模型的研究者和开发者
当前最先进的生成模型在图像合成或文本生成方面表现卓越，但因其模型规模庞大，难以在智能手机上部署。量化方法通过降低参数精度（如使用 exttt{INT8}）实现高效计算，然而激进量化仍难以保持模型质量。为此，本文提出一种新型分数比特量化（FraQAT）方法：将模型精度从32位逐步降至4位，并在优化过程中利用分数比特以维持高质量生成。实验表明，该方法在多个扩散模型（包括SD3.5-Medium、Sana、 exttt{PixArt}、FLUX.1-schnell）上均取得更优性能，相比标准QAT实现4%-7%更低的FiD值。最终，我们在三星S25U手机上成功部署并运行Sana模型，其基于高通SM8750-AB Snapdragon 8 Elite Hexagon Tensor Processor（HTP）硬件。
原文摘要 · Abstract (English)
State-of-the-art (SOTA) generative models have demonstrated impressive capabilities in image synthesis or text generation, often with a large capacity model. However, these large models cannot be deployed on smartphones due to the limited availability of on-board memory and computations. Quantization methods lower the precision of the model parameters, allowing for efficient computations, \eg, in \INT{8}. Although aggressive quantization addresses efficiency and memory constraints, preserving the quality of the model remains a challenge. To retain quality in previous aggressive quantization, we propose a new fractional bits quantization (\short) approach. The novelty is a simple yet effective idea: we progressively reduce the model's precision from 32 to 4 bits per parameter, and exploit the fractional bits during optimization to maintain high generation quality. We show that the \short{} yields improved quality on a variety of diffusion models, including SD3.5-Medium, Sana, \pixart, and FLUX.1-schnell, while achieving $4-7\%$ lower FiD than standard QAT. Finally, we deploy and run Sana on a Samsung S25U, which runs on the Qualcomm SM8750-AB Snapdragon 8 Elite Hexagon Tensor Processor (HTP).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。