2比特大模型量化新方法，显著提升精度与稳定性
R2Q: Towards Robust 2-Bit Large Language Models via Residual Refinement Quantization
- 将2比特量化拆分为两次1比特子量化，构建自适应量化网格
- 在多个大模型上实现优于现有2比特方法的性能，尤其在细粒度任务中
- 模块化设计兼容现有训练框架，适合部署极限压缩场景
大型语言模型的快速发展带来了巨大的计算和内存需求，推动了低比特量化技术的应用。尽管8比特和4比特量化已广泛使用，但将量化扩展到2比特仍面临严重精度下降的挑战。为此，我们提出残差精炼量化（R2Q）——一种新型2比特量化框架，将量化过程分解为两个顺序的1比特子量化，形成自适应量化网格。在Llama、OPT和Qwen等多个模型上，涵盖问答、常识推理和语言建模的多样化基准测试表明，R2Q在细粒度和粗粒度设置下均持续优于现有2比特量化方法。通过残差学习机制进行量化精炼，R2Q提升了性能，改善了训练稳定性，并在极端压缩下加速收敛。此外，其模块化设计可无缝集成至现有量化感知训练（QAT）框架中。
原文摘要 · Abstract (English)
The rapid progress of Large Language Models (LLMs) has brought substantial computational and memory demands, spurring the adoption of low-bit quantization. While 8-bit and 4-bit formats have become prevalent, extending quantization to 2 bits remains challenging due to severe accuracy degradation. To address this, we propose Residual Refinement Quantization (R2Q)-a novel 2-bit quantization framework that decomposes the process into two sequential 1-bit sub-quantizations, forming an adaptive quantization lattice. Extensive evaluations on Llama, OPT, and Qwen across diverse benchmarks-covering question answering, commonsense reasoning, and language modeling-demonstrate that R2Q consistently outperforms existing 2-bit quantization methods in both fine-grained and coarse-grained settings. By refining quantization through a residual learning mechanism, R2Q enhances performance, improves training stability, and accelerates convergence under extreme compression. Furthermore, its modular design enables seamless integration with existing quantization-aware training (QAT) frameworks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。