用可学习的格子码本压缩大模型，低比特下仍保持高精度。
Learning Grouped Lattice Vector Quantizers for Low-Bit LLM Compression
- 为每组权重设计自适应格子码本，提升低比特压缩效率。
- 在GSM、LAMBADA等数据集上，4比特时准确率优于现有方法。
- 适合资源受限场景部署大模型，如移动端或边缘设备。
大型语言模型虽表现卓越，但推理需大量计算与内存。后训练量化（PTQ）可通过低比特存储降低资源需求，但标准均匀量化常导致性能显著下降，尤其在低比特情况下。本文提出分组格子向量量化（GLVQ）框架，为每组权重分配由可学习生成矩阵定义的定制化格子码本。针对量化过程不可导问题，采用Babai近似法实现近似最近格点搜索，支持生成矩阵的稳定优化。训练完成后，解码仅需矩阵向量乘法，流程高效实用。在多个基准测试中，该方法在模型大小与精度之间取得更优平衡，显著优于现有后训练量化基线，证明其在严苛资源约束下部署大模型的有效性。代码已开源：https://github.com/xzhang9308/GLVQ。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated remarkable capabilities but typically require extensive computational resources and memory for inference. Post-training quantization (PTQ) can effectively reduce these demands by storing weights in lower bit-width formats. However, standard uniform quantization often leads to notable performance degradation, particularly in low-bit scenarios. In this work, we introduce a Grouped Lattice Vector Quantization (GLVQ) framework that assigns each group of weights a customized lattice codebook, defined by a learnable generation matrix. To address the non-differentiability of the quantization process, we adopt Babai rounding to approximate nearest-lattice-point search during training, which enables stable optimization of the generation matrices. Once trained, decoding reduces to a simple matrix-vector multiplication, yielding an efficient and practical quantization pipeline. Experiments on multiple benchmarks show that our approach achieves a better trade-off between model size and accuracy compared to existing post-training quantization baselines, highlighting its effectiveness in deploying large models under stringent resource constraints. Our source code is available on GitHub repository: https://github.com/xzhang9308/GLVQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。