提出新型非均匀量化方法，提升大模型推理速度与精度。
CubicQuant: Parametric Non-Uniform Codebooks for High-Throughput LLM Inference with 1-8-Bit Weights
- 用参数化立方曲线实现非均匀量化，保持密集整数编码流
- 在1-8比特下显著降低重建误差，尤其对拉普拉斯分布效果明显
- 支持直接GPU打包执行，适合高吞吐量大模型推理场景
大型语言模型推理中的权重量化需在自适应重建精度与高效GPU执行之间取得平衡。传统均匀整数量化限制每组为线性网格，低比特浮点格式采用固定指数-尾数结构，而学习型码本虽灵活但解码不规则且需额外元数据。本文提出CubicQuant，一种参数化非均匀标量格式，在保持密集整数码流的同时，使每组权重的重建水平可调。通过由两个形状参数和一个缩放参数定义的单调立方曲线，将等间距幅度码映射为非均匀级别。该格式覆盖1-8比特权重载荷，包含对称均匀整数量化作为特例，每权重有效宽度为B + 64/G比特（载荷宽B，分组大小G）。推导了在均匀、高斯和拉普拉斯分布下的群体失真，提出了连续及动态A8载体感知拟合目标，并描述了直接打包权重的GPU执行方案。在G=128、每分布15,360样本条件下，W4 CubicQuant相较最优截断四比特均匀整数量化，重建均方根误差分别降低3.90%（均匀）、13.49%（高斯）、28.14%（拉普拉斯）；相较最佳枚举四比特有限浮点格式，降幅为3.90%、9.44%、6.27%。初步H200内核测量显示：窄GEMV任务中模型类型执行更快，随着行数增加，动态A8逐渐占优。结果验证了该格式的表征潜力与直接可执行性；下游模型质量及跨设备端到端性能仍待进一步评估。
原文摘要 · Abstract (English)
Weight quantization for large-language-model inference must balance adaptive reconstruction levels with representations regular enough for efficient GPU execution. Uniform integers constrain each group to a linear grid. Low-bit floating-point formats use a fixed exponent-mantissa structure, while learned codebooks gain flexibility at the cost of irregular decoding and additional metadata. We introduce CubicQuant, a parametric non-uniform scalar format that preserves a dense integer code stream while adapting reconstruction levels within each weight group. A monotonic cubic curve, specified by two shape parameters and one scale, maps uniformly spaced magnitude codes to non-uniform levels. The family spans 1-8-bit weight payloads, contains symmetric uniform integer quantization as an exact special case, and has effective width B + 64/G bits per weight for payload width B and group size G. We derive population distortion under Uniform, Gaussian, and Laplace distributions, formulate continuous and Dynamic-A8-carrier-aware fitting objectives, and describe direct packed-weight GPU execution. For finite groups of G=128 with 15,360 samples per distribution, W4 CubicQuant reduced reconstruction RMSE relative to optimally clipped four-bit uniform integer quantization by 3.90% on Uniform, 13.49% on Gaussian, and 28.14% on Laplace samples. Relative to the best enumerated four-bit finite floating-point format, the reductions were 3.90%, 9.44%, and 6.27%. Preliminary H200 kernel measurements show a workload-dependent crossover: model-dtype execution is faster for narrow GEMV, while Dynamic A8 becomes favorable as row count grows. The results establish the format's representational promise and direct executability; downstream model quality and cross-device end-to-end performance remain open evaluation questions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。