提出新量化方法，在低比特下保持大模型性能。
OJBKQ: Objective-Joint Babai-Klein Quantization
- 将权重量化建模为激活与权重联合优化问题。
- 3-4比特下困惑度优于现有方法，计算开销相当。
- 适合需要高效压缩大模型的研究者使用。
后训练量化（PTQ）被广泛用于在不重新训练的情况下压缩大语言模型。然而，许多现有的仅权重量化方法依赖启发式目标和贪心舍入，导致在低比特量化时出现明显性能下降。本文提出 OJBKQ（Objective-Joint Babai-Klein Quantization with K-Best Sampling），一种逐层的 PTQ 方法，将权重量化建模为激活与权重的联合优化问题。该建模在每层中形成一个带箱约束的整数最小二乘（BILS）问题，属于 NP 难问题。针对权重矩阵的每一列，我们采用扩展的 Babai 最近平面算法和 Klein 的随机化 Babai 算法，以寻找最小残差的 Babai-Klein 点，作为 BILS 问题的次优解。在大语言模型上的实验结果表明，与现有 PTQ 方法相比，OJBKQ 在 3-4 比特下实现了更低的困惑度，同时保持了相近的计算成本。
原文摘要 · Abstract (English)
Post-training quantization (PTQ) is widely used to compress large language models without retraining. However, many existing weight-only methods rely on heuristic objectives and greedy rounding, thus leading to noticeable degradation under low-bit quantization. In this work, we introduce OJBKQ (Objective-Joint Babai-Klein Quantization with K-Best Sampling), a layer-wise PTQ method that formulates weight quantization as a joint optimization problem over activations and weights. This formulation results in a multiple-right-hand-side box-constrained integer least squares (BILS) problem in each layer, which is NP-hard. For each column of the weight matrix, we apply an extended Babai nearest-plane algorithm and an extended version of Klein's randomized Babai algorithm to find the minimum-residual Babai-Klein point, a sub-optimal solution to the BILS problem. Experimental results on large language models show that OJBKQ achieves lower perplexity at 3-4 bits compared to existing PTQ approaches, while maintaining comparable computational cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。