通过调整权重空间提升低比特量化稳定性，解决模型压缩中精度下降难题
HeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioning
- 引入可学习的旋转压缩矩阵重塑损失曲面，降低海森矩阵最大特征值
- 在W3A16极端低比特下使Llama3 8B的GSM8K准确率达70.15%，避免逻辑崩溃
- 无需修改架构，可无缝接入现有压缩流程，适合部署场景下的高效模型压缩
后训练量化（PTQ）作为主流模型压缩技术，常出现‘低误差、高损失’悖论，根源在于大语言模型损失曲面中少数高曲率方向对扰动极度敏感。为此，我们提出赫斯曼鲁棒量化（HeRo-Q）算法，通过在量化前对权重空间施加轻量级可学习的旋转-压缩矩阵，联合重构损失曲面，有效降低海森矩阵的最大特征值，显著增强对量化噪声的鲁棒性。该方法无需架构修改，计算开销极小，可无缝集成至现有PTQ流程。在Llama与Qwen模型上的实验表明，HeRo-Q不仅在标准W4A8设置下优于GPTQ、AWQ和SpinQuant等先进方法，更在极具挑战性的W3A16超低比特场景中表现卓越，将Llama3 8B在GSM8K上的准确率提升至70.15%，有效避免了激进量化常见的逻辑崩溃问题。
原文摘要 · Abstract (English)
Post Training Quantization (PTQ), a mainstream model compression technique, often leads to the paradoxical 'low error, high loss' phenomenon because it focuses solely on minimizing quantization error. The root cause lies in the Hessian matrix of the LLM loss landscape: a few high curvature directions are extremely sensitive to perturbations. To address this, we propose the Hessian Robust Quantization (HeRo Q) algorithm, which applies a lightweight, learnable rotation-compression matrix to the weight space prior to quantization. This joint framework reshapes the loss landscape by reducing the largest Hessian eigenvalue and reducing its max eigenvalue, thereby significantly enhancing robustness to quantization noise. HeRo-Q requires no architectural modifications, incurs negligible computational overhead, and integrates seamlessly into existing PTQ pipelines. Experiments on Llama and Qwen models show that HeRo Q consistently outperforms state of the art methods including GPTQ, AWQ, and SpinQuant not only achieving superior performance under standard W4A8 settings, but also excelling in the highly challenging W3A16 ultra low bit regime, where it boosts GSM8K accuracy on Llama3 8B to 70.15\% and effectively avoids the logical collapse commonly seen in aggressive quantization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。