提出统一框架ParetoQ，实现1-4比特低比特量化性能突破。
ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization
- 构建统一框架，支持1到4比特量化对比
- 3比特以上模型保持原始分布，2比特以下大幅改变表征
- 2比特模型性能超4比特，600M模型胜过3B原模型
在极低比特量化的模型尺寸与精度权衡中，最优比特宽度始终存在争议。尽管有研究主张4比特最优，也有观点认为1.58比特更优，但缺乏跨比特位数的统一比较框架，使结论不够稳固。本文提出ParetoQ，首个支持1比特、1.58比特、2比特、3比特和4比特量化设置的统一框架。实验发现，在2比特与3比特间存在显著学习过渡：3比特及以上微调模型仍贴近预训练分布，而2比特及以下模型表征发生剧烈变化。通过优化训练策略与量化函数，ParetoQ超越所有针对特定比特位设计的方法。令人惊讶的是，其三值化600M参数模型在精度上超过此前最先进三值化3B模型，仅用五分之一参数。大量实验证明，三值、2比特和3比特量化在尺寸-精度权衡上表现相当，普遍优于4比特和二值量化。考虑到硬件限制，2比特量化在内存压缩与加速方面具有巨大潜力。
原文摘要 · Abstract (English)
The optimal bit-width for achieving the best trade-off between quantized model size and accuracy has been a subject of ongoing debate. While some advocate for 4-bit quantization, others propose that 1.58-bit offers superior results. However, the lack of a cohesive framework for different bits has left such conclusions relatively tenuous. We present ParetoQ, the first unified framework that facilitates rigorous comparisons across 1-bit, 1.58-bit, 2-bit, 3-bit, and 4-bit quantization settings. Our findings reveal a notable learning transition between 2 and 3 bits: For 3-bits and above, the fine-tuned models stay close to their original pre-trained distributions, whereas for learning 2-bit networks or below, the representations change drastically. By optimizing training schemes and refining quantization functions, ParetoQ surpasses all previous methods tailored to specific bit widths. Remarkably, our ParetoQ ternary 600M-parameter model even outperforms the previous SoTA ternary 3B-parameter model in accuracy, using only one-fifth of the parameters. Extensive experimentation shows that ternary, 2-bit, and 3-bit quantization maintains comparable performance in the size-accuracy trade-off and generally exceeds 4-bit and binary quantization. Considering hardware constraints, 2-bit quantization offers promising potential for memory reduction and speedup.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。