无需校准和变换，实现4比特权重量化，显著降低部署成本。
Calibration and Transformation-Free Weight-Only LLMs Quantization via Dynamic Grouping
- 动态分组优化，自适应生成多尺度量化级别。
- 4比特块级量化下，WikiText-2上困惑度8.43，接近全精度7.81。
- 纯CPU实现，无需校准数据或中间转换，适合资源受限场景。
大语言模型性能强大，但部署时受内存与算力限制。低比特后训练量化（PTQ）是可行方向，但通常依赖校准数据、辅助变换及GPU工具。为此，我们提出MSB（多尺度二值化），一种无需校准和变换的PTQ方法，将二值量化推广至多比特设置。MSB通过最小化组内方差的动态分组准则，生成跨粒度（从张量到每行64元素块）的一致多尺度量化层级，无需校准或中间变换。量化步骤在基于CPU的求解器中实现，评估采用标准bfloat16执行，不使用低比特打包。在Llama 3.2 3B模型上，MSB在4比特权重仅量化块级配置下，于WikiText-2上达到8.43的困惑度，相比全精度7.81略有下降，优于默认设置下的GPTQ（12.23）。整体上，MSB为低比特PTQ提供新优化视角，并简化了部署流程。
原文摘要 · Abstract (English)
Large Language Models (LLMs) deliver strong performance but are difficult to deploy under tight memory and compute constraints. Low-bit post-training quantization (PTQ) is a promising direction; however, it typically relies on calibration data, auxiliary transformations, and GPU tools. To address these limitations, we propose MSB (Multi Scale Binary), a calibration-free and transformation-free PTQ method that generalizes binary quantization to multi-bit settings. MSB optimizes a dynamic grouping criterion that minimizes within group variance, yielding group-wise multiscale levels that can be applied consistently across granularities from per tensor to block-wise configurations with 64 elements groups per row, without calibration or intermediate transforms. We implement the optimization in a CPU based solver for the quantization step and evaluate using standard bfloat16 execution without low-bit packing. On Llama 3.2 3B, MSB achieves 8.43 perplexity on WikiText-2 under 4-bit weight only block-wise quantization, compared to 7.81 in full precision and 12.23 with GPTQ its default setup. Overall, MSB provides a new optimization perspective for low-bit PTQ while simplifying the pipeline by removing calibration and transformations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。