针对实时视频超分模型压缩难题，提出分层敏感量化方法
LSGQuant: Layer-Sensitivity Guided Quantization for One-Step Diffusion Real-World Video Super-Resolution
- 根据各层敏感度动态调整量化策略
- 在保持接近原始性能的同时显著降低计算开销
- 适合部署在资源受限的实时视频处理场景
一步扩散模型在真实世界视频超分辨率任务中展现出强大能力与快速推理优势。然而，扩散变压器（DiTs）模型规模庞大、计算成本高，限制了实际应用。尽管低比特量化是模型压缩的常用手段，但输入潜在表示的高动态范围和各层行为差异仍导致量化效果下降。为此，本文提出LSGQuant，一种面向一步扩散式真实世界VSR的分层敏感量化方法。该方法引入动态范围自适应量化器（DRAQ）以适配视频令牌激活值；通过校准阶段分析层内统计特性，估计各层敏感度，并采用方差导向的层训练策略（VOLTS）；同时引入量化感知优化（QAO），联合优化量化分支与保留的高精度分支。大量实验表明，本方法在几乎保持原始模型性能的前提下，显著优于现有量化技术。代码已公开：https://github.com/zhengchen1999/LSGQuant。
原文摘要 · Abstract (English)
One-Step Diffusion Models have demonstrated promising capability and fast inference in video super-resolution (VSR) for real-world. Nevertheless, the substantial model size and high computational cost of Diffusion Transformers (DiTs) limit downstream applications. While low-bit quantization is a common approach for model compression, the effectiveness of quantized models is challenged by the high dynamic range of input latent and diverse layer behaviors. To deal with these challenges, we introduce LSGQuant, a layer-sensitivity guided quantizing approach for one-step diffusion-based real-world VSR. Our method incorporates a Dynamic Range Adaptive Quantizer (DRAQ) to fit video token activations. Furthermore, we estimate layer sensitivity and implement a Variance-Oriented Layer Training Strategy (VOLTS) by analyzing layer-wise statistics in calibration. We also introduce Quantization-Aware Optimization (QAO) to jointly refine the quantized branch and a retained high-precision branch. Extensive experiments demonstrate that our method has nearly performance to origin model with full-precision and significantly exceeds existing quantization techniques. Code is available at: https://github.com/zhengchen1999/LSGQuant.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。