通过多粒度分析实现图像超分动态量化,提升移动端模型效率与精度。
Thinking in Granularity: Dynamic Quantization for Image Super-Resolution by Intriguing Multi-Granularity Clues
- 基于局部块的多粒度特征分析,自适应分配比特位数。
- 在多个SR模型上优于当前最先进方法,保持高保真重建效果。
- 不依赖层敏感性,适合资源受限场景下的高效部署。
动态量化在图像超分辨率(SR)中受到越来越多关注,它使重型SR模型能在移动设备上运行并保持优异性能。现有方法根据局部区域差异调整层到比特的配置,自适应分配每层和每块的比特数。然而,这些方法在SR精度与量化效率之间仍存在权衡。此外,单独为每层调整量化级别会破坏原有的层间关系,削弱量化模型的表征能力。本文提出Granular-DQ,利用图像固有特性,摒弃以往对层敏感性的依赖。该方法对局部块进行多粒度分析,深入探索其信息密度,实现一种块级独立、层无关的动态量化范式。具体而言,设计粒度-比特控制器(GBC),捕捉不同块从粗到细的粒度表示,按其对整图的贡献比例决定比特分配。在此基础上,研究比特宽度与信息密度的关系,提出熵到比特(E2B)机制,进一步精细化高比特块的动态适配。大量实验证明,Granular-DQ在多种SR模型上均优于近期最先进方法,具备优异的泛化能力。代码与补充说明见:https://github.com/MmmingS/Granular-DQ.git。
原文摘要 · Abstract (English)
Dynamic quantization has attracted rising attention in image super-resolution (SR) as it expands the potential of heavy SR models onto mobile devices while preserving competitive performance. Existing methods explore layer-to-bit configuration upon varying local regions, adaptively allocating the bit to each layer and patch. Despite the benefits, they still fall short in the trade-off of SR accuracy and quantization efficiency. Apart from this, adapting the quantization level for each layer individually can disturb the original inter-layer relationships, thus diminishing the representation capability of quantized models. In this work, we propose Granular-DQ, which capitalizes on the intrinsic characteristics of images while dispensing with the previous consideration for layer sensitivity in quantization. Granular-DQ conducts a multi-granularity analysis of local patches with further exploration of their information densities, achieving a distinctive patch-wise and layer-invariant dynamic quantization paradigm. Specifically, Granular-DQ initiates by developing a granularity-bit controller (GBC) to apprehend the coarse-to-fine granular representations of different patches, matching their proportional contribution to the entire image to determine the proper bit-width allocation. On this premise, we investigate the relation between bit-width and information density, devising an entropy-to-bit (E2B) mechanism that enables further fine-grained dynamic bit adaption of high-bit patches. Extensive experiments validate the superiority and generalization ability of Granular-DQ over recent state-of-the-art methods on various SR models. Code and supplementary statement can be found at \url{https://github.com/MmmingS/Granular-DQ.git}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。