arXiv:2601.21069eess.IV2026-01

旋转量化有效关键在范围压缩,而非零点聚集。

Any Orthogonal Transform Will Do: Range Reduction Explains Rotation-Based Quantization for Super-Resolution

  • 通过正交变换压缩权重与激活值范围,提升量化精度。
  • 在45种配置下旋转量化均优于传统方法,3~4位时提升0.267比特/参数。
  • 任意正交矩阵均可实现效果,哈达玛矩阵优势来自其结构设计。

基于旋转的量化(如用哈达玛矩阵乘以权值和激活值后再量化)已成为图像超分辨率中的标准做法,但其有效性机制尚未明确。本文通过配对假设检验,分析了多种正交变换(包括Sylvester哈达玛、QuIP#-Paley、DCT-II及随机正交矩阵)对模型性能的影响。结果显示:范围压缩在所有变换下均能复现并迁移至状态空间模型与卷积网络;分布更接近正态性也具一致性;而零附近质量增加仅出现在零填充的Sylvester构造中,属于填充效应而非变换本质。在三个缩放因子、三种位宽和五个基准测试组成的45个配置下,旋转量化在所有情况下均优于同位宽的非旋转量化器,且替换为DCT-II或随机正交矩阵仅损失0.06–0.07 dB,低于哈达玛矩阵所含的填充成本。因此,哈达玛矩阵的优势源于其±1取值与O(n log n)蝶形结构。研究提出CompSRT方案,在44/45配置中优于2DQuant,仅在2/3位时因填充代价未胜出;而CondiQuant始终领先0.03–1.40 dB。

原文摘要 · Abstract (English)

Rotation-based quantization -- multiplying weights and activations by a Hadamard matrix before quantizing -- is now standard, but why it works is explained informally, via incoherence, the central limit theorem, or kurtosis, and rarely tested directly. We ask which property of the post-rotation distribution drives the gain, using paired hypothesis tests on the weights and activations of an image super-resolution transformer. Three candidate properties separate cleanly. Range reduction replicates under every orthogonal transform we tried -- Sylvester Hadamard, QuIP#-Paley, DCT-II, and random orthogonal matrices -- and transfers to a state-space backbone and to CNN weights. Improved normality also replicates. The increased mass near zero, by contrast, is significant only under the zero-padded Sylvester construction and disappears under unpadded transforms, so we attribute it to the padding, not the transform. The end task follows the same pattern: over a grid of three scale factors, three bitwidths and five benchmarks, rotation improves PSNR in all 45 configurations against an otherwise identical un-rotated quantizer at the same nominal bitwidth (the rotated arm carries 0.267 bits/parameter of Sylvester padding at 4 bits and stays ahead in every cell when charged for it), while substituting DCT-II or a random orthogonal matrix costs only 0.06-0.07 dB, less than the padding the Hadamard is charged for. The choice of kernel is therefore an implementation decision: the Hadamard's advantage is its +/-1 entries and O(n log n) butterflies. We package the recipe as CompSRT, which improves on 2DQuant in 44 of 45 configurations at equal nominal bitwidth -- charged for its padding that edge holds at 3 and 2 bits but not at 4. CondiQuant remains ahead at every operating point by 0.03-1.40 dB, which we report.

量化旋转超分辨率正交变换

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。