提出新型量化方法，让小模型在极低精度下仍保持高保真度。
HAS-VQ: Hessian-Adaptive Sparse Vector Quantization for High-Fidelity LLM Compression
- 用二阶敏感度分离关键参数，避免均匀量化对尾部分布的破坏
- 4.23有效比特下困惑度14.23，优于标准INT4的20.03
- 相比浮点模型压缩2.3倍，困惑度几乎无损，适合带宽受限场景
后训练量化对在资源受限设备上部署大语言模型至关重要。然而，标准整数量化（如INT4）会因对重尾分布权重参数施加均匀网格而显著降低性能，尤其在小型模型（<20亿参数）中更为明显。我们提出HAS-VQ（Hessian-Adaptive Sparse Vector Quantization），一种压缩框架，通过二阶敏感性分析严格将高敏感异常值与主体权重分布解耦。HAS-VQ采用赫斯曼掩码解耦策略隔离敏感参数，随后对剩余密集部分进行鲁棒向量量化（VQ）。关键创新在于引入残差稀疏反馈机制，校正最敏感维度的量化误差，确保异常值精确重建。我们在SmolLM2-1.7B上评估HAS-VQ，展现出两种优势：(1) 非支配性优势：在4.23有效比特/参数（BPP）下，困惑度达14.23，显著优于标准INT4基线（4.71 BPP时困惑度20.03）；(2) 高保真压缩：相较于FP16基线，模型大小减少2.3倍（7.03 BPP），困惑度仅从10.04降至10.12，统计上无差异，为带宽受限环境提供近乎无损压缩方案。代码开源于https://github.com/VladimerKhasia/HASVQ。
原文摘要 · Abstract (English)
Post-training quantization is essential for deploying Large Language Models (LLMs) on resource-constrained devices. However, standard integer quantization (e.g., INT4) fundamentally degrades performance by imposing a uniform grid on the heavy-tailed distribution of weight parameters, particularly in smaller-scale models (e.g., <2B parameters). We introduce HAS-VQ (Hessian-Adaptive Sparse Vector Quantization), a compression framework that strictly decouples high-sensitivity outliers from the bulk weight distribution using second-order sensitivity analysis. HAS-VQ employs a Hessian-Masked Decoupling strategy to isolate sensitive parameters, followed by robust Vector Quantization (VQ) of the remaining dense body. Crucially, we introduce a residual sparse feedback mechanism that corrects quantization errors in the most sensitive dimensions, ensuring exact reconstruction of outliers. We evaluate HAS-VQ on SmolLM2-1.7B, demonstrating two distinct regimes of superiority: (1) Pareto Dominance over Integer Baselines: At 4.23 effective bits-per-parameter (BPP), we achieve a perplexity of 14.23, significantly outperforming the standard INT4 baseline (20.03 PPL at 4.71 BPP). (2) High-Fidelity Compression: Relative to the FP16 baseline, HAS-VQ achieves a 2.3x reduction in model size (7.03 BPP) while maintaining statistically indistinguishable perplexity (10.12 vs. 10.04), effectively offering a lossless compression alternative for bandwidth-constrained environments. The code is available at https://github.com/VladimerKhasia/HASVQ
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。