arXiv:2606.01412cs.LGcs.IT2026-06

提出一种无需训练的低精度量化方法,显著提升模型压缩后的性能。

GPTQ-intrinsic LoRA: A Near-optimal Algorithm for Low-precision Quantization with Low-rank Adaptation

论文配图:GPTQ-intrinsic LoRA: A Near-optimal Algorithm for Low-precision Quantization with Low-rank Adaptation
图 1 · 摘自论文原文
  • 将低秩补偿直接嵌入GPTQ量化流程,通过校准矩阵优化修正项。
  • 理论证明误差上限与信息论下界接近,依赖剩余秩而非全矩阵范数。
  • 适用于大模型压缩,尤其适合语言和视觉模型的高精度量化部署。

后训练量化广泛用于压缩大型神经网络,但激进的低位量化会显著降低模型质量。常见做法是通过低秩修正项对量化权重进行补充,形式为 $W\approx Q+LR$。本文从逐层重构目标 $ |XW-X(Q+LR) |_F^2$ 出发研究该问题,其中 $X$ 为校准矩阵。首次建立了在有限字母表与有界低秩补偿约束下的信息论下界。随后提出 GPTQ-intrinsic LoRA,一种无需训练的算法,通过调整校准海森矩阵将低秩修正直接融入 GPTQ 流程。当选择 $L=V_r$(即 $X$ 的前 $r$ 个右奇异向量)时,证明了逐层重构误差上界,其中原本依赖于 $ |X |_F^2$ 的项被秩-$r$ 剩余项 $ |X-X_r |_F^2$ 替代,仅含正则化项。在自然结构假设下,该上界与其主导尺度的下界一致,仅差常数与温和因子。此外引入 Bid-Up,一种固定网格量化精修步骤,可与最优低秩补偿交替使用,并保证逐层重构误差非增。在 Qwen3 语言模型与 DeiT 视觉变换器上的实验表明,GPTQ-intrinsic LoRA 优于 GPTQ 及其后续低秩修正,且精修循环带来额外增益。

原文摘要 · Abstract (English)

Post-training quantization is widely used for compressing large neural networks, but aggressive low-bit quantization can significantly degrade model quality. A common remedy is to augment the quantized weights with a low-rank correction, leading to approximations of the form $W\approx Q+LR$. In this paper, we study this low-precision plus low-rank representation through the layer-wise reconstruction objective $\|XW-X(Q+LR)\|_F^2$, where $X$ is a calibration matrix. We establish, to our knowledge, the first information-theoretic lower bounds for this problem under finite-alphabet and bounded low-rank compensation constraints. We then propose GPTQ-intrinsic LoRA, a training-free algorithm that incorporates the low-rank correction directly into a GPTQ-style quantization pass by appropriately augmenting the calibration Hessian. For the choice $L=V_r$, where $V_r$ contains the top right singular vectors of $X$, we prove layer-wise reconstruction error bounds in which the usual GPTQ dependence on $\|X\|_F^2$ is replaced by the rank-$r$ residual $\|X-X_r\|_F^2$, up to regularization terms. Under natural structural assumptions, these bounds match the information-theoretic lower bounds in their dominant scaling, up to constants and mild factors. We also introduce Bid-Up, a fixed-grid quantization refinement step that can be alternated with optimal low-rank compensation with guaranteed non-increasing layer-wise reconstruction error. Experiments on Qwen3 language models and DeiT vision transformers show that GPTQ-intrinsic LoRA improves over GPTQ and GPTQ followed by low-rank compensation, with additional gains from refinement loops.

量化低秩适配GPTQ大模型压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。