提出一种无需零点的符号对称量化，提升低比特模型精度且不增加推理开销。
Signed Symmetric Quantization for Few-Bit Integers
- 通过符号选择规则将额外负数位分配给主异常值尾部，保持零点为零。
- 在4比特下比非对称量化内存减少9%，吞吐量提升2.45倍，精度更高。
- 适用于大语言模型低比特部署，尤其适合追求高效推理的场景。
整数表示中负数可表示的数量比正数多一个。但标准对称量化器通常设为正比例因子，导致多余数值归于负尾部，迫使正异常值截断。本文发现，在低比特下，这种截断是显著的量化误差来源。非对称量化虽能通过零点偏移解决此问题，但会带来运行时开销。例如在AMD EPYC「Turin」CPU上，4比特对称格式相比非对称格式内存减少高达9%，吞吐量提升2.45倍。本文提出符号对称量化作为第三种选择：在保持对称量化运行效率的同时，利用基于符号的绝对最大值网格，将额外可表示值分配到主导异常值尾部，并保持零点为零。理论分析表明，该方法在ℓ₂量化误差下为条件最优，且在88%-99%的预训练大语言模型权重组中满足条件。进一步证明，将标准对称量化器的比例因子取负，等价于在相同符号整数字母表上进行单位零点偏移。在Qwen3、Qwen3.5和Llama3系列模型上的实证验证显示，相较标准无符号对称量化，在不增加推理成本的前提下，困惑度与少样本下游准确率均有提升。
原文摘要 · Abstract (English)
The signed integer alphabet contains one more negative representable value than positive. Yet, by convention, the standard symmetric integer quantizer fixes its scale to be strictly positive, which assigns this extra representable value to the negative tail and can force clipping of positive outliers. In this work, we show that, at few-bit precision, such clipping is a non-trivial source of quantization error. Asymmetric quantization addresses this problem with a zero point, shifting the grid toward the observed data range; however, this flexibility is well-known to carry a runtime penalty. For example, in llama.cpp on an AMD EPYC(TM) "Turin" CPU, a 4-bit symmetric format uses up to 9% less memory with up to 2.45$\times$ higher throughput than its asymmetric counterpart. We highlight signed symmetric quantization as a third option that retains the runtime profile of symmetric quantization without the penalty of the asymmetric format: our signed absmax grid places the extra representable value on the dominant-outlier tail through a principled and lightweight sign selection rule while keeping the zero point at zero. Our theoretical analysis offers two main results. First, we establish the signed absmax grid as conditionally bound-optimal on $\ell_2$ quantization error, and show that the condition holds for 88-99% of weight groups across pre-trained large language models (LLMs) at low bit widths. Second, we show that negating the scale of a standard symmetric quantizer is analytically equivalent to a unit zero point shift on the same signed integer alphabet. We empirically validate our proposal on models from the Qwen3, Qwen3.5, and Llama3 families, and observe improvement in perplexity and downstream few-shot accuracy over the standard unsigned symmetric quantizer at no extra inference cost
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。