针对自回归视觉生成模型的量化难题,提出无需训练的高效压缩方法。
PTQ4ARVG: Post-Training Quantization for AutoRegressive Visual Generation Models
- 设计三阶段无训练量化框架,解决通道、令牌和样本级分布问题。
- 8位和6位量化后性能接近原模型,适用于部署场景。
- 适合研究或应用自回归视觉生成模型的开发者参考使用。
自回归视觉生成(ARVG)模型在架构上与语言模型兼容,同时性能可媲美基于扩散的模型。量化常用于神经网络以减小模型规模和计算延迟,但将其应用于ARVG仍鲜有研究,现有方法难以有效泛化。本文分析发现三大挑战:(1)通道级严重异常值;(2)令牌级高度动态激活;(3)样本级分布不匹配。为此,提出PTQ4ARVG——一种无需训练的后训练量化(PTQ)框架:(1)增益投影缩放(GPS)通过泰勒展开量化缩放增益,推导最优缩放因子以缓解通道异常;(2)静态令牌级量化(STWQ)利用ARVG固有的固定令牌长度与位置不变分布特性,无动态校准开销地处理令牌波动;(3)分布引导校准(DGC)选取对分布熵贡献最大的样本,消除样本级分布偏差。大量实验表明,PTQ4ARVG可将ARVG模型有效量化至8位和6位,且保持优异性能。代码已开源:http://github.com/BienLuky/PTQ4ARVG。
原文摘要 · Abstract (English)
AutoRegressive Visual Generation (ARVG) models retain an architecture compatible with language models, while achieving performance comparable to diffusion-based models. Quantization is commonly employed in neural networks to reduce model size and computational latency. However, applying quantization to ARVG remains largely underexplored, and existing quantization methods fail to generalize effectively to ARVG models. In this paper, we explore this issue and identify three key challenges: (1) severe outliers at channel-wise level, (2) highly dynamic activations at token-wise level, and (3) mismatched distribution information at sample-wise level. To these ends, we propose PTQ4ARVG, a training-free post-training quantization (PTQ) framework consisting of: (1) Gain-Projected Scaling (GPS) mitigates the channel-wise outliers, which expands the quantization loss via a Taylor series to quantify the gain of scaling for activation-weight quantization, and derives the optimal scaling factor through differentiation.(2) Static Token-Wise Quantization (STWQ) leverages the inherent properties of ARVG, fixed token length and position-invariant distribution across samples, to address token-wise variance without incurring dynamic calibration overhead.(3) Distribution-Guided Calibration (DGC) selects samples that contribute most to distributional entropy, eliminating the sample-wise distribution mismatch. Extensive experiments show that PTQ4ARVG can effectively quantize the ARVG family models to 8-bit and 6-bit while maintaining competitive performance. Code is available at http://github.com/BienLuky/PTQ4ARVG .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。