通过覆盖异常通道优化校准，显著提升低比特量化模型性能。
Coverage-Based Calibration for Post-Training Quantization via Weighted Set Cover over Outlier Channels
- 将校准样本选择建模为加权异常通道覆盖问题，优先激活高活跃维度。
- 在128样本下使MMLU提升1.2-1.5分，困惑度损失降低15%-30%。
- 算法高效无需GPU，适合小样本校准场景，尤其适合资源受限部署。
后训练量化（PTQ）通过少量校准集将大语言模型压缩至低比特，其效果高度依赖样本选择。我们发现一种失效模式：校准样本未能激活异常通道（隐藏维度中异常大的激活值），导致量化器低估动态范围，引发主导层损失的通道级重建误差。为此，我们提出校准质量更应由加权异常通道覆盖率决定，而非泛化代表性，并将样本选择建模为加权集合覆盖问题。该目标函数单调子模，贪婪算法COVERCAL基于预计算激活统计，无需额外GPU时间。进一步证明权重选择内在一致：在简化截断模型下，遗漏的加权覆盖率可上界代理损失，验证了目标的理论合理性。在LLaMA-2、LLaMA-3和Mistral模型上，覆盖多种下游任务与AWQ/GPTQ后端，COVERCAL优于随机、最大困惑度、最大激活方差和分层基线，在小校准预算下提升最显著。在INT4+128样本时，相较随机校准提升MMLU 1.2–1.5分，困惑度下降15%–30%；64样本时已达到甚至超过256样本随机校准效果。贡献在于校准选择的新范式，而非新量化后端，提供高效算法与代理损失支撑的理论基础。
原文摘要 · Abstract (English)
Post-Training Quantization (PTQ) compresses large language models to low bit-widths using a small calibration set, and its quality depends strongly on which samples are chosen. We identify a failure mode in which calibration samples fail to activate outlier channels, hidden dimensions with unusually large activations, causing the quantizer to underestimate their dynamic range and producing per-channel reconstruction errors that dominate layer-wise loss. Motivated by this observation, we argue that PTQ calibration quality is governed more by weighted outlier-channel coverage than by generic sample representativeness, and formulate calibration selection as a weighted set cover problem over outlier channels. The objective is monotone submodular, and the greedy algorithm, COVERCAL, operates on pre-computed activation statistics and requires no GPU time at selection. We further show that the weight choice is internally consistent: under a stylized clipping model, missed weighted coverage upper-bounds surrogate loss, justifying the weighted coverage objective as principled rather than purely empirical. Across LLaMA-2, LLaMA-3, and Mistral, under AWQ and GPTQ backends and five downstream evaluations, COVERCAL improves over random, max-perplexity, max-activation-variance, and stratified baselines, with the largest gains at small calibration budgets. At INT4 with 128 samples, COVERCAL improves MMLU by 1.2 to 1.5 points over random calibration and reduces perplexity degradation by 15 to 30\%; with 64 samples, it matches or exceeds random calibration at 256. The contribution is not a new PTQ backend but a formulation of calibration selection as weighted outlier coverage, with a simple, efficient algorithm and a surrogate-based justification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。