提出对数空间量化方法，4比特下性能优于传统线性量化。
$\text{Log}_\text{b}$Quant: Quantizing Language Models in Logarithmic Space
- 在对数空间中设计可调基底的量化方式，适配参数分布
- 4比特精度下在多个基准上超越非对称线性量化
- 适合消费级显卡私有部署，兼顾内存节省与推理速度
量化已成为降低现代语言模型内存占用和推理延迟的关键技术，尤其有助于在消费级设备和边缘设备上部署。以往工作主要聚焦于均匀量化码本，但这类方法因低频高幅值权重易导致表示不佳。本文提出Log$_\text{b}$Quant，一种具有可调基底的新型对数量化方法，以适应常见的参数分布。实验表明，该方法在4比特精度下，相比张量级粒度的非对称线性量化，在多个性能基准上表现更优，同时实现适度加速和显著内存节省，适用于消费级GPU的私有部署。
原文摘要 · Abstract (English)
Quantization has become an invaluable tool to reduce memory requirements and inference speed of modern language models, in particular to make them available for consumer setups and edge devices. While previous work has primarily focused on uniform quantization codebooks, such approaches are prone to suboptimal representations due to low-frequency high-magnitude weights. We introduce Log$_\text{b}$Quant, a novel logarithmic quantization approach with adjustable bases, to adapt to common parameter distributions. We show that our method exhibits superior performance at 4-bit precision on several performance benchmarks compared to asymmetric linear quantization at tensor-wise granularity, while achieving moderate speedup and high memory savings, making it suitable for private use on consumer-grade GPUs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。