提出RUQuant方法,让大模型量化更准更快,无需微调。
RUQuant: Towards Refining Uniform Quantization for Large Language Models
- 通过正交变换重构激活值分布,解决量化区间偏移问题。
- 130亿参数模型用W6A6量化后准确率达全精度99.8%,仅需一分钟。
- 适合追求高效部署的大模型应用,尤其在资源受限场景下。
大语言模型规模增大带来了部署效率挑战,尤其是在资源受限环境下。后训练量化(PTQ)成为无需重训练即可压缩模型的实用方案。现有方法多采用权重与激活值的均匀量化,但因激活值分布非均匀,常导致精度大幅下降。本文从Lloyd-Max最优性条件出发,发现量化区间内激活值分布不均是导致最优量化点偏离中点的根本原因。为此提出两阶段正交变换方法RUQuant:第一阶段将激活值分块,利用由Householder反射和Givens旋转构造的复合正交矩阵映射至均匀采样目标向量;第二阶段使用全局Householder反射,基于Transformer输出差异进一步优化以最小化量化误差。实验表明,该方法无需模型微调即可实现近最优性能:对130亿参数模型,W6A6量化达全精度99.8%准确率,W4A4达97%,耗时约一分钟;微调变体性能更优,验证了方法的有效性与可扩展性。
原文摘要 · Abstract (English)
The increasing size and complexity of large language models (LLMs) have raised significant challenges in deployment efficiency, particularly under resource constraints. Post-training quantization (PTQ) has emerged as a practical solution by compressing models without requiring retraining. While existing methods focus on uniform quantization schemes for both weights and activations, they often suffer from substantial accuracy degradation due to the non-uniform nature of activation distributions. In this work, we revisit the activation quantization problem from a theoretical perspective grounded in the Lloyd-Max optimality conditions. We identify the core issue as the non-uniform distribution of activations within the quantization interval, which causes the optimal quantization point under the Lloyd-Max criterion to shift away from the midpoint of the interval. To address this issue, we propose a two-stage orthogonal transformation method, RUQuant. In the first stage, activations are divided into blocks. Each block is mapped to uniformly sampled target vectors using composite orthogonal matrices, which are constructed from Householder reflections and Givens rotations. In the second stage, a global Householder reflection is fine-tuned to further minimize quantization error using Transformer output discrepancies. Empirical results show that our method achieves near-optimal quantization performance without requiring model fine-tuning: RUQuant achieves 99.8% of full-precision accuracy with W6A6 and 97% with W4A4 quantization for a 13B LLM, within approximately one minute. A fine-tuned variant yields even higher accuracy, demonstrating the effectiveness and scalability of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。