统一二值编码与均匀量化，提升大模型压缩精度
Unifying Uniform and Binary-coding Quantization for Accurate Compression of Large Language Models
- 融合均匀量化灵活映射与二值编码非均匀分层优势
- 在GSM8K上最高比现有方法高4.60%准确率
- 无需额外开销，适合高效部署大模型
如何在保持精度的前提下对大语言模型进行量化压缩？量化是高效部署大语言模型的关键。二值编码量化（BCQ）和均匀量化（UQ）是两种有前景的量化方案，分别具备强表达能力和良好可优化性，但均未能兼顾两者优势。本文提出UniQuanF（统一灵活映射量化），通过融合UQ中的灵活映射机制与BCQ的非均匀量化层级，同时实现强表达与可优化性。我们设计了统一初始化策略，并引入局部与周期性映射技术以精确优化参数。优化后，通过统一定理消除计算与内存开销，使模型在不增加部署成本的前提下发挥优异精度。实验表明，UniQuanF显著优于现有UQ与BCQ方法，在GSM8K基准上最高提升4.60%准确率。
原文摘要 · Abstract (English)
How can we quantize large language models while preserving accuracy? Quantization is essential for deploying large language models (LLMs) efficiently. Binary-coding quantization (BCQ) and uniform quantization (UQ) are promising quantization schemes that have strong expressiveness and optimizability, respectively. However, neither scheme leverages both advantages. In this paper, we propose UniQuanF (Unified Quantization with Flexible Mapping), an accurate quantization method for LLMs. UniQuanF harnesses both strong expressiveness and optimizability by unifying the flexible mapping technique in UQ and non-uniform quantization levels of BCQ. We propose unified initialization, and local and periodic mapping techniques to optimize the parameters in UniQuanF precisely. After optimization, our unification theorem removes computational and memory overhead, allowing us to utilize the superior accuracy of UniQuanF without extra deployment costs induced by the unification. Experimental results demonstrate that UniQuanF outperforms existing UQ and BCQ methods, achieving up to 4.60% higher accuracy on GSM8K benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。