GPTAQ无需微调即可高效压缩大模型，显著降低量化误差。
GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration
- 采用非对称校准，逐层匹配全精度模型输出，减少误差累积。
- 在405B语言模型和EVA-02视觉模型上实现高精度低比特量化。
- 仅需20行额外代码，单卡即可完成超大规模模型压缩。
我们提出GPTAQ，一种用于压缩大规模Transformer架构的新型无微调量化方法。与以往GPTQ方法独立校准各层不同，GPTAQ始终将量化层输出与全精度模型输出精确匹配，形成所谓非对称校准。该方案有效减少了前序层累积的量化误差。我们基于最优大脑压缩理论推导出闭式解，显式最小化量化误差及累积不对称误差。此外，通过通道并行、神经元分解和乔列斯基重构等技术实现求解并行化。结果表明，GPTAQ仅比GPTQ多20行代码，却在低比特量化下性能更优。令人瞩目的是，在单张GPU上，我们成功量化了405B语言模型及排名第一的视觉模型EVA-02（预训练图像分类准确率达90%）。代码已开源于Github。
原文摘要 · Abstract (English)
We introduce GPTAQ, a novel finetuning-free quantization method for compressing large-scale transformer architectures. Unlike the previous GPTQ method, which independently calibrates each layer, we always match the quantized layer's output to the exact output in the full-precision model, resulting in a scheme that we call asymmetric calibration. Such a scheme can effectively reduce the quantization error accumulated in previous layers. We analyze this problem using optimal brain compression to derive a close-formed solution. The new solution explicitly minimizes the quantization error as well as the accumulated asymmetry error. Furthermore, we utilize various techniques to parallelize the solution calculation, including channel parallelization, neuron decomposition, and Cholesky reformulation for matrix fusion. As a result, GPTAQ is easy to implement, simply using 20 more lines of code than GPTQ but improving its performance under low-bit quantization. Remarkably, on a single GPU, we quantize a 405B language transformer as well as EVA-02, the rank first vision transformer that achieves 90% pretraining Imagenet accuracy. Code is available at Github.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。