2比特统一量化框架，兼顾精度与速度。
UniSVQ: 2-bit Unified Scalar-Vector Quantization
- 将标量与向量量化统一为整数格点的仿射变换结构
- 在多个LLM上实现优于现有标量量化、接近向量量化性能
- 适合需要低延迟部署的大模型应用
训练后2比特量化可实现大语言模型（LLMs）的低成本部署与推理加速。标量量化（SQ）和向量量化（VQ）是两种主要方法，前者存在显著性能下降，后者则带来计算与存储开销。我们提出UniSVQ，一种统一的2比特量化框架，通过将码字参数化为整数格点的仿射变换，连接标量与向量量化。该结构保持与优化整数内核的兼容性，同时保留了VQ的灵活性。我们进一步引入数据驱动的分块微调策略，直接最小化量化重构误差。在多个LLM家族和零样本基准上的大量实验表明，UniSVQ持续优于最先进标量量化方法，并达到与先进向量量化相当的性能，同时提供更高的推理吞吐量。代码已公开于https://github.com/AI9Stars/UniSVQ。
原文摘要 · Abstract (English)
Post-training quantization at the 2-bit level enables low-cost deployment and inference acceleration for large language models (LLMs). Scalar quantization (SQ) and vector quantization (VQ) are two primary quantization methods, however, the former suffers from significant performance degradation, and the latter incurs computational and storage overhead. We propose UniSVQ, a unified 2-bit quantization framework that bridges scalar and vector quantization by parameterizing codewords as an affine transform of integer lattices. This structure preserves compatibility with optimized integer kernels while retaining much of VQ's flexibility. We further introduce a data-driven block-wise fine-tuning strategy to directly minimize quantization reconstruction error. Extensive experiments across multiple LLM families and zero-shot benchmarks demonstrate that UniSVQ consistently outperforms state-of-the-art SQ methods and achieves performance comparable to advanced VQ methods, while providing higher inference throughput. Codes are publicly available at https://github.com/AI9Stars/UniSVQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。