提出无需零点的两比特量化方法,提升旋转后LLM推理效率与精度。
Qift: Shift-Friendly No-Zero W2 Post-Training Quantization for Rotated W2A4/KV4 LLM Inference

- 基于哈达玛旋转重构权重分布,设计无零点固定码本
- 在224个线性模块上,纯两比特与混合精度下均降低困惑度
- 适合部署轻量级模型,无需训练或复杂代码本
两比特权重量化可显著提升大模型推理的内存效率,但标准的W2级别集{-2,-1,0,+1}在激进的W2A4/KV4设置下常出现性能崩溃。本文研究了哈达玛旋转量化流水线中两比特权重的标量级别集几何结构。传统非对称W2性能显著优于标准级别集,表明失败不仅源于位宽限制,更与重建级别有关。在LLaMA-2-7B和LLaMA-3.1-8B的每个模型中224个线性模块上,预训练权重已接近零中心化,而哈达玛旋转主要使标准化权重分布趋于高斯化:峰度和Q-Q误差下降数个数量级。基于此近似零中心高斯源模型,提出Qift——一种用于旋转后W2A4/KV4推理的固定无零点两比特级别集。主级别集为{±0.5, ±1.5},等价于半尺度重参数化下的{±1, ±3};幂次为二的变体使用{±1, ±4}以支持符号与移位解码。Qift重新设计固定两比特编码到级别的映射,无需训练、无学习码本、无分组网格、无零点,保留标准逐通道缩放。比例分析揭示有效内/外质心比范围为0.25至0.33,解释为何镜像无零(MNZ)、Lloyd、NF2和PoT-MNZ表现良好,而{±1, ±2}不成立。在两个模型上,无零级别集在纯两比特和分层混合精度下持续降低困惑度、提升下游准确率,并改善GPTQ残差行为。在第16层混合精度设置下,大幅缩小与W3A4的差距,同时保持一半变压器层为两比特精度,提供一种简单、源感知且部署友好的替代方案。
原文摘要 · Abstract (English)
Two-bit weight quantization is attractive for memory-efficient LLM inference, but the standard W2 level set {-2,-1,0,+1} often collapses under aggressive W2A4/KV4 settings. We study the scalar level-set geometry of two-bit weights in a Hadamard-rotated quantization pipeline. Conventional asymmetric W2 substantially improves over the standard level set, indicating that W2A4 failure is not only a bit-width problem but also a reconstruction-level problem. Across all 224 linear modules in each of LLaMA-2-7B and LLaMA-3.1-8B, pretrained weights are already nearly zero-centered, while Hadamard rotation primarily Gaussianizes their standardized shape: excess kurtosis and Q-Q error drop by orders of magnitude. Based on this approximate zero-centered Gaussian-like source model, we propose Qift, a fixed no-zero W2 level set for rotated W2A4/KV4 inference. The main level set is {+/-0.5, +/-1.5}, equivalently {+/-1, +/-3} under a half-scale reparameterization; a power-of-two variant uses {+/-1, +/-4} for sign-and-shift decoded weight application. Qift redesigns the fixed two-bit code-to-level mapping and is training-free, learned-codebook-free, group-grid-free, and zero-point-free, retaining the standard per-channel scale. A scale-invariant ratio analysis identifies an effective inner/outer centroid ratio range of 0.25 to 0.33, explaining why mirror no-zero (MNZ), Lloyd, NF2, and PoT-MNZ perform well while {+/-1, +/-2} does not. On both models, the no-zero level sets consistently improve pure W2A4 perplexity, L-layer mixed W2/W4 perplexity, downstream accuracy, and GPTQ residual behavior over the standard W2 level set. At L=16 mixed precision, they substantially narrow the gap to W3A4 while keeping half of the transformer layers at two-bit precision, giving a simple, source-aware, and deployment-friendly alternative to more complex learned W2 codebooks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。