arXiv:2602.01273cs.CV2026-02中稿 · ICML被引 2

针对扩散模型超分任务,提出首个专用量化框架,兼顾细节与效率。

Q-DiT4SR: Exploration of Detail-Preserving Diffusion Transformer Quantization for Real-World Image Super-Resolution

  • 设计分层低秩分解与动态精度分配策略,适配扩散变换器结构
  • 在W4A4下实现5.8倍压缩、6.14倍加速,纹理保持最优
  • 适合部署高保真图像超分的轻量化场景,如移动端实时处理

近期,扩散变换器(DiTs)在真实世界图像超分(Real-ISR)中展现出生成高质量纹理的能力,但其庞大的推理开销阻碍了实际部署。尽管训练后量化(PTQ)是加速的有效方案,现有方法多聚焦于U-Net架构,而通用的DiT量化通常针对文生图任务设计。直接应用于基于DiT的超分模型会导致局部纹理严重退化。为此,本文提出首个专为DiT-based Real-ISR设计的PTQ框架Q-DiT4SR。我们提出H-SVD,一种分层奇异值分解,在统一参数预算下融合全局低秩分支与局部块级秩1分支。进一步提出方差感知时空混合精度(VaSMP),基于率失真理论无数据地分配跨层权重位宽,同时通过动态规划(DP)调度层内激活精度,仅需极少量校准。在多个真实世界数据集上的实验表明,所提Q-DiT4SR在W4A6和W4A4设置下均达到当前最优性能。特别地,W4A4配置将模型大小压缩5.8倍,计算量减少6.14倍。代码与模型将公开于https://github.com/xunzhang1128/Q-DiT4SR。

原文摘要 · Abstract (English)

Recently, Diffusion Transformers (DiTs) have emerged in Real-World Image Super-Resolution (Real-ISR) to generate high-quality textures, yet their heavy inference burden hinders real-world deployment. While Post-Training Quantization (PTQ) is a promising solution for acceleration, existing methods in super-resolution mostly focus on U-Net architectures, whereas generic DiT quantization is typically designed for text-to-image tasks. Directly applying these methods to DiT-based super-resolution models leads to severe degradation of local textures. Therefore, we propose Q-DiT4SR, the first PTQ framework specifically tailored for DiT-based Real-ISR. We propose H-SVD, a hierarchical SVD that integrates a global low-rank branch with a local block-wise rank-1 branch under a matched parameter budget. We further propose Variance-aware Spatio-Temporal Mixed Precision: VaSMP allocates cross-layer weight bit-widths in a data-free manner based on rate-distortion theory, while VaTMP schedules intra-layer activation precision across diffusion timesteps via dynamic programming (DP) with minimal calibration. Experiments on multiple real-world datasets demonstrate that our Q-DiT4SR achieves SOTA performance under both W4A6 and W4A4 settings. Notably, the W4A4 quantization configuration reduces model size by 5.8$\times$ and computational operations by 6.14$\times$. Our code and models will be available at https://github.com/xunzhang1128/Q-DiT4SR.

图像超分扩散模型模型量化细节保留

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。