通过无数据旋转优化，显著减少大模型权重重塑时的异常值。
OptRot: Mitigating Weight Outliers via Data-Free Rotations for Post-Training Quantization
- 用旋转最小化权重四次方，无须数据即可学习有效变换。
- 在W4A8下优于哈达玛旋转和依赖数据的方法，提升量化精度。
- 适合追求高效低比特量化、无需训练数据的部署场景。
大型语言模型（LLMs）的权重重塑和激活中存在异常值，使量化困难。近期工作利用旋转缓解此问题。本文提出方法，通过最小化可衡量且低成本的代理目标来学习可融合的旋转，以逼近权重量化误差。主要聚焦于GPTQ量化方法。核心方法OptRot仅通过最小化旋转后权重的逐元素四次方来减少权重异常值。实验表明，OptRot在权重量化上优于哈达玛旋转及更昂贵的数据依赖方法如SpinQuant和OSTQuant。同时在W4A8设置下改善了激活量化表现。还提出数据依赖方法OptRot⁺，通过引入激活协方差信息进一步提升性能。但在W4A4设置下，两者表现均下降，揭示了权重与激活量化间的权衡。
原文摘要 · Abstract (English)
The presence of outliers in Large Language Models (LLMs) weights and activations makes them difficult to quantize. Recent work has leveraged rotations to mitigate these outliers. In this work, we propose methods that learn fusible rotations by minimizing principled and cheap proxy objectives to the weight quantization error. We primarily focus on GPTQ as the quantization method. Our main method is OptRot, which reduces weight outliers simply by minimizing the element-wise fourth power of the rotated weights. We show that OptRot outperforms both Hadamard rotations and more expensive, data-dependent methods like SpinQuant and OSTQuant for weight quantization. It also improves activation quantization in the W4A8 setting. We also propose a data-dependent method, OptRot$^{+}$, that further improves performance by incorporating information on the activation covariance. In the W4A4 setting, we see that both OptRot and OptRot$^{+}$ perform worse, highlighting a trade-off between weight and activation quantization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。