arXiv:2509.09679cs.LGcs.AI2025-09被引 8

用可学习的蝴蝶变换实现2比特大模型量化,显著提升低精度部署性能。

ButterflyQuant: Ultra-low-bit LLM Quantization through Learnable Orthogonal Butterfly Transforms

  • 引入可学习的蝴蝶变换替代固定正交变换,适配不同层的激活分布。
  • 仅用128个校准样本,单卡几分钟内完成训练,实现2比特量化且保持高精度。
  • 适合资源受限场景下的大模型部署,尤其对极端低比特量化有显著优势。

大型语言模型内存占用巨大,严重限制在消费级硬件上的部署。量化通过降低数值精度减少内存开销,但极端2比特量化因激活值中的异常值导致性能急剧下降。基于旋转的方法如QuIP和QuaRot在量化前使用正交变换消除异常值,利用计算不变性:$\mathbf{y} = \mathbf{Wx} = (\mathbf{WQ}^T)(\mathbf{Qx})$($\mathbf{Q}$为正交矩阵)。然而这些方法采用固定变换——哈达玛矩阵实现最优最坏情况相干性 $μ= 1/\sqrt{n}$——无法适应特定权重分布。我们发现不同Transformer层具有不同的异常值模式,因此提出应采用分层自适应旋转而非统一方案。本文提出ButterflyQuant,用可学习的蝴蝶变换替代哈达玛旋转,参数化为连续的Givens旋转角。与哈达玛矩阵离散的$\{+1, -1\}$元素不可导不同,蝴蝶变换的连续参数化支持梯度优化,同时构造保证正交性。该正交约束确保异常值抑制的理论保障,并实现$O(n \log n)$复杂度,仅需$\frac{n \log n}{2}$个可学习参数。我们进一步引入后变换激活的均匀性正则化,促进更平滑的分布以利于量化。学习仅需128个校准样本,单卡几分钟内收敛。

原文摘要 · Abstract (English)

Large language models require massive memory footprints, severely limiting deployment on consumer hardware. Quantization reduces memory through lower numerical precision, but extreme 2-bit quantization suffers from catastrophic performance loss due to outliers in activations. Rotation-based methods such as QuIP and QuaRot apply orthogonal transforms to eliminate outliers before quantization, using computational invariance: $\mathbf{y} = \mathbf{Wx} = (\mathbf{WQ}^T)(\mathbf{Qx})$ for orthogonal $\mathbf{Q}$. However, these methods use fixed transforms--Hadamard matrices achieving optimal worst-case coherence $μ= 1/\sqrt{n}$--that cannot adapt to specific weight distributions. We identify that different transformer layers exhibit distinct outlier patterns, motivating layer-adaptive rotations rather than one-size-fits-all approaches. In this work, we propose ButterflyQuant, which replaces Hadamard rotations with learnable butterfly transforms parameterized by continuous Givens rotation angles. Unlike Hadamard's discrete $\{+1, -1\}$ entries that are non-differentiable and thus prohibit gradient-based learning, butterfly transforms' continuous parameterization enables smooth optimization while guaranteeing orthogonality by construction. This orthogonal constraint ensures theoretical guarantees in outlier suppression while achieving $O(n \log n)$ computational complexity with only $\frac{n \log n}{2}$ learnable parameters. We further introduce a uniformity regularization on post-transformation activations to promote smoother distributions amenable to quantization. Learning requires only 128 calibration samples and converges in minutes on a single GPU.

大模型量化2比特量化可学习变换低比特部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。