精准优化量化缩放因子，让大模型压缩后更准确。
Optimal Post-Training Quantization Scales and Where to Find Them
- 提出分段最优缩放算法，用校准数据精确计算每通道缩放值。
- 在不同模型和比特位宽下，困惑度与零样本准确率均显著提升。
- 适合追求高精度模型压缩的工程师和研究者使用。
后训练量化（PTQ）通过将权重映射到低比特表示来压缩大型语言模型。通常采用无需数据的启发式方法选择定义量化网格的缩放因子。本文提出PiSO（分段缩放优化）算法，利用校准数据在四舍五入量化下精确高效地计算每通道权重缩放因子。PiSO将缩放搜索空间划分为有限个区间，每个区间内目标函数存在闭式解。我们通过合理启发式方法将PiSO扩展至组级量化，并提出有效的缩放优化与误差校正交替策略。在Llama和Qwen系列模型上，针对多种模型规模和目标权重比特位宽的实验表明，该方法在独立使用或结合误差校正时，均持续提升困惑度与下游零样本准确率；尤其在低比特、高挑战性量化场景中收益更明显。
原文摘要 · Abstract (English)
Post-training quantization (PTQ) compresses large language models by mapping weights to low-bit representations. The scaling factor that defines the quantization grid is typically chosen using simple, data-free heuristics. In this work, we present PiSO (Piecewise Scale Optimization), an algorithm that leverages calibration data to compute the optimal channel-wise weight scales exactly and efficiently under round-to-nearest quantization. PiSO partitions the scale search space into finitely many intervals on which the objective admits a closed-form minimizer. We extend PiSO to group-wise quantization via principled heuristics and propose effective strategies for interleaving scale optimization with error correction. Experiments on Llama and Qwen models across multiple model sizes and target weight bit-widths demonstrate consistent improvements in perplexity and downstream zero-shot accuracy, both standalone and combined with error correction. In particular, we observe increased benefits as the target bit-width narrows and quantization becomes more challenging.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。