arXiv:2602.05902cs.LGcs.AI2026-02被引 2

无需训练的量化框架,自动修正层间误差,提升大模型部署精度。

CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantization

  • 基于几何分解设计自适应修正系数,避免逐层过拟合。
  • 在多个大模型上实现更低困惑度和更高下游任务准确率。
  • 无需调参,适合快速部署低比特大模型的工程场景。

后训练量化(PTQ)通过小规模校准集将预训练权重映射到低比特格式,实现大语言模型高效部署,但逐层量化引入层间失配:前层量化误差改变后层输入分布,使激活偏离全精度模型。现有方法虽引入失配感知校准目标,却未解决各层应如何调整校准目标的问题。完全应用修正易过拟合有限校准数据,固定系数缩放又忽略各层失配估计的可靠性差异。为此,我们提出CoreQ,一种无学习的PTQ框架,其通过失配的几何分解推导闭式系数,自适应调节各层修正强度,减少对校准数据的过拟合且无需超参数调优。给定修正目标后,CoreQ采用高效的贪心逐次舍入求解器,并结合可选的边界束搜索扩展K-CoreQ,在适度增加计算量下进一步提升性能。在多类大模型、尺度、比特位宽与量化设置下,CoreQ在困惑度与下游准确率上均优于强基线。

原文摘要 · Abstract (English)

Post-training quantization (PTQ) enables efficient deployment of large language models by mapping pretrained weights to low-bit formats without retraining, typically using a small calibration set to minimize a layer-wise calibration objective. However, this sequential procedure induces a mismatch: errors from earlier quantized layers alter the inputs received by later layers, causing the activations to deviate from those of the full-precision model. Recent approaches introduce mismatch-aware calibration objectives to compensate for this effect, but leave open how much of the observed mismatch should shift each layer's calibration target. Fully applying this correction can overfit limited calibration data, while scaling the mismatch correction with a fixed coefficient ignores varying reliability of mismatch estimates across layers. To address these limitations, we propose CoreQ, a learning-free PTQ framework that applies a closed-form coefficient for mismatch correction derived from a geometric decomposition of the mismatch. The resulting coefficient adapts the correction across layers, reduces overfitting to finite calibration data, and requires no hyperparameter tuning. Given the corrected target, CoreQ minimizes the induced triangular least-squares objective with an efficient greedy successive-rounding solver and a bounded beam-search extension, K-CoreQ, that trades modest additional compute for improved performance. Across multiple LLM families, scales, bit-widths, and quantization settings, CoreQ improves perplexity and downstream accuracy over strong PTQ baselines.

量化大模型PTQ无训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。