arXiv:2508.03485cs.CV2025-08被引 8

提出新量化方法,让扩散模型在极低精度下仍保持高质量图像视频生成能力。

LRQ-DiT: Log-Rotation Post-Training Quantization of Diffusion Transformers for Image and Video Generation

  • 采用对数映射分配更多量化区间给权重密集区,减少误差。
  • 动态旋转机制自适应处理激活值中的异常点,提升量化稳定性。
  • 适用于资源受限场景下的图像与视频生成模型压缩,效果显著。

扩散变压器(DiTs)在文本到图像和文本到视频生成任务中表现优异,但其高计算开销和大参数量限制了在资源受限环境中的应用。后训练量化(PTQ)是降低内存占用和加速推理的有前景方案,但现有方法在极端低比特设置下性能严重下降。通过实验分析,我们识别出两个关键障碍:(1) DiT 模型权重呈类高斯分布且尾部较长,导致均匀量化分配不均,引发显著量化误差;该现象普遍存在于不同 DiT 模型的线性层权重中,严重制约性能。(2) 存在两类激活异常值:(i) 轻度异常值,数值略高;(ii) 显著异常值,集中在特定通道且幅度巨大,破坏激活量化。为此,我们提出 LRQ-DiT,一种高效精准的后训练量化框架。首先引入双对数量化(TLQ),基于对数映射将更多量化区间分配至中间密集区域,实现与权重分布对齐,降低误差。其次提出自适应旋转方案(ARS),根据激活波动动态选择哈达玛或异常值感知旋转,有效缓解两类异常值影响。在多个文本到图像及文本到视频的 DiT 模型上进行大量实验,结果表明 LRQ-DiT 在极低比特下仍能保持高生成质量。

原文摘要 · Abstract (English)

Diffusion Transformers (DiTs) have achieved impressive performance in text-to-image and text-to-video generation. However, their high computational cost and large parameter sizes pose significant challenges for usage in resource-constrained scenarios. Effective compression of models has become a crucial issue that urgently needs to be addressed. Post-training quantization (PTQ) is a promising solution to reduce memory usage and accelerate inference, but existing PTQ methods suffer from severe performance degradation under extreme low-bit settings. After experiments and analysis, we identify two key obstacles to low-bit PTQ for DiTs: (1) the weights of DiT models follow a Gaussian-like distribution with long tails, causing uniform quantization to poorly allocate intervals and leading to significant quantization errors. This issue has been observed in the linear layer weights of different DiT models, which deeply limits the performance. (2) two types of activation outliers in DiT models: (i) Mild Outliers with slightly elevated values, and (ii) Salient Outliers with large magnitudes concentrated in specific channels, which disrupt activation quantization. To address these issues, we propose LRQ-DiT, an efficient and accurate post-training quantization framework for image and video generation. First, we introduce Twin-Log Quantization (TLQ), a log-based method that allocates more quantization intervals to the intermediate dense regions, effectively achieving alignment with the weight distribution and reducing quantization errors. Second, we propose an Adaptive Rotation Scheme (ARS) that dynamically applies Hadamard or outlier-aware rotations based on activation fluctuation, effectively mitigating the impact of both types of outliers. Extensive experiments on various text-to-image and text-to-video DiT models demonstrate that LRQ-DiT preserves high generation quality.

模型量化扩散模型图像生成轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。