arXiv:2605.05693cs.AIcs.LG2026-05被引 1

通过关注重要权重,让大模型量化后更接近原模型,提升推理效果。

Saliency-Aware Regularized Quantization Calibration for Large Language Models

论文配图:Saliency-Aware Regularized Quantization Calibration for Large Language Models
图 1 · 摘自论文原文
  • 在量化校准中加入权重偏离惩罚项,控制新权重与原始权重距离。
  • 在多个大模型上测试,量化后困惑度和零样本准确率均有提升。
  • 可无缝接入现有方法,不增加推理开销,适合部署场景。

后训练量化(PTQ)是缓解大语言模型(LLM)内存与延迟约束的有效方法。现有多数PTQ方法通过最小化特定校准数据上的逐层重构误差来确定量化参数,通常采用缩放搜索或基于格拉姆的方法。然而,从泛化风险角度,仅依赖有限或非代表性校准数据的重建误差目标可能使量化权重偏离原始浮点权重,进而影响下游性能。为此,我们提出正则化量化校准(RQC),在标准PTQ目标中引入显式控制权重偏离的正则项。进一步推广该框架,引入显著性感知正则项,形成显著性感知正则化量化校准(SARQC)。该正则项促使量化权重在校准过程中保持靠近原始权重,从而提升推理时的泛化能力。SARQC可无缝集成至现有PTQ流程,并统一适用于基于缩放搜索与基于格拉姆的方法。在密集型与混合专家(MoE)LLM上的大量实验表明,其在不引入额外推理开销的前提下,持续提升了困惑度与零样本准确率。

原文摘要 · Abstract (English)

Post-training quantization (PTQ) is an effective approach for deploying large language models (LLMs) under memory and latency constraints. Most existing PTQ methods determine quantization parameters by minimizing a layer-wise reconstruction error on a predetermined calibration dataset, typically optimized via either scale search or Gram-based methods. However, from the perspective of generalization risk, existing PTQ calibration objectives based solely on empirical reconstruction error over limited or unrepresentative calibration data may move the quantized weights away from the original floating-point weights, potentially degrading downstream performance. To address this issue, we propose \emph{Regularized Quantization Calibration} (RQC), a unified framework that augments standard PTQ objectives with a regularizer that explicitly controls weight deviation from the original weights. We further generalize this framework to incorporate a saliency-aware regularizer, resulting in \emph{Saliency-Aware Regularized Quantization Calibration} (SARQC). The proposed regularization encourages quantized weights to remain close to the original weights during calibration, leading to improved generalization at inference time. SARQC integrates seamlessly into existing PTQ pipelines and enhances both scale-search-based and Gram-based methods under a unified formulation. Extensive experiments on dense and Mixture-of-Experts LLMs demonstrate consistent improvements in perplexity and zero-shot accuracy, without introducing additional inference overhead.

大模型量化正则化推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。