基于不确定性优化的量化框架，提升模型压缩精度。
BayesQ: Uncertainty-Guided Bayesian Quantization
- 用后验分布指导量化，最小化预期损失
- 3.0/3.5/4.0比特下显著优于现有方法
- 适合追求高精度低比特部署的研究者
我们提出 BayesQ，首个在后验期望损失下优化的后训练量化框架。BayesQ 在权重上拟合轻量级高斯后验分布（默认对角拉普拉斯；可选 K-FAC/低秩），通过后验协方差进行白化，设计代码本以最小化后验期望失真，并使用贪心背包算法在全局比特预算下分配混合精度，最大化每比特的期望损失降低。对于标量量化，后验期望均方误差可得闭式表格；任务感知代理通过小规模校准集上的短蒙特卡洛处理。可选的仅校准蒸馏将量化模型与后验预测教师对齐。在平均每位权重 3.0/3.5/4.0 比特下，BayesQ 在 ResNet-50（ImageNet）和 BERT-base（GLUE）上优于强基线，例如相比 GPTQ 在 RN50 上提升 +1.5/+0.7/+0.3 个百分点，在 BERT 上提升 +1.1/+0.4/+0.2 分，且仅需一次预处理，相当于一次 GPTQ 通路。BayesQ 将低比特量化重构为实际后训练流程中的不确定性感知风险最小化。
原文摘要 · Abstract (English)
We present BayesQ, an uncertainty-guided post-training quantization framework that is the first to optimize quantization under the posterior expected loss. BayesQ fits a lightweight Gaussian posterior over weights (diagonal Laplace by default; optional K-FAC/low-rank), whitens by the posterior covariance, designs codebooks to minimize posterior-expected distortion, and allocates mixed precision via a greedy knapsack that maximizes marginal expected-loss reduction per bit under a global budget. For scalar quantizers, posterior-expected MSE yields closed-form tables; task-aware proxies are handled by short Monte Carlo on a small calibration set. An optional calibration-only distillation aligns the quantized model with the posterior predictive teacher. At matched average bits/weight of 3.0/3.5/4.0, BayesQ improves over strong PTQ baselines on ResNet-50 (ImageNet) and BERT-base (GLUE) e.g., vs. GPTQ by $+1.5/+0.7/+0.3$ top-1 percentage points on RN50 and $+1.1/+0.4/+0.2$ GLUE points on BERT, while requiring one-time preprocessing comparable to a GPTQ pass. BayesQ reframes low-bit quantization as uncertainty-aware risk minimization in a practical, post-training pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。