提出无需数据的量化方法，让扩散Transformer在低比特下保持高性能。
DiTAS: Quantizing Diffusion Transformers via Enhanced Activation Smoothing
- 通过时序平滑技术缓解激活值通道异常，降低量化误差。
- 实现4比特权重、8比特激活的量化，性能接近全精度模型。
- 无需微调，适合移动端等资源受限场景部署。
扩散Transformer（DiTs）因在视觉生成任务中表现优于传统U-Net结构的扩散模型而受到广泛关注。然而，其更高性能伴随着更大的参数量和部署成本，限制了在移动设备等资源受限场景的应用。本文提出DiTAS，一种无需数据的后训练量化（PTQ）方法，用于高效推理DiT。该方法利用提出的时序聚合平滑技术，缓解输入激活值中通道级异常对量化的影响，显著降低极低比特下的量化误差。为进一步提升量化模型性能，采用逐层网格搜索优化平滑因子，并引入免训练的LoRA模块进行权重量化，通过交替优化最小化量化误差，无需额外微调。实验结果表明，所提方法可在保持与全精度模型相当性能的前提下，实现DiT的4比特权重、8比特激活（W4A8）量化。
原文摘要 · Abstract (English)
Diffusion Transformers (DiTs) have recently attracted significant interest from both industry and academia due to their enhanced capabilities in visual generation, surpassing the performance of traditional diffusion models that employ U-Net. However, the improved performance of DiTs comes at the expense of higher parameter counts and implementation costs, which significantly limits their deployment on resource-constrained devices like mobile phones. We propose DiTAS, a data-free post-training quantization (PTQ) method for efficient DiT inference. DiTAS relies on the proposed temporal-aggregated smoothing techniques to mitigate the impact of the channel-wise outliers within the input activations, leading to much lower quantization error under extremely low bitwidth. To further enhance the performance of the quantized DiT, we adopt the layer-wise grid search strategy to optimize the smoothing factor. Moreover, we integrate a training-free LoRA module for weight quantization, leveraging alternating optimization to minimize quantization errors without additional fine-tuning. Experimental results demonstrate that our approach enables 4-bit weight, 8-bit activation (W4A8) quantization for DiTs while maintaining comparable performance as the full-precision model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。