提出QUASAR方法,用在线感知重建降低量化训练的损失下限,提升低比特模型性能。
QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction

- 在训练中实时进行轻量级损失感知重建,避免权重更新与损失计算不一致
- 2-4比特下相比现有方法降低至少10%的KL散度,2比特时准确率提升3.5-4.3个百分点
- 无需修改推理格式,兼容整数量化和NVFP4,适合部署优化
随着大语言模型推理向低精度演进,后训练量化(PTQ)变得愈发脆弱,量化感知训练(QAT)成为保持模型质量的关键。然而,传统QAT使用有损重构的全精度权重计算损失与代理梯度,却对潜空间权重进行更新,导致训练轨迹次优且损失下限较高。二阶PTQ虽通过最小化损失感知重构误差缓解该问题,但固定模型下的重构需数小时,随权重演化重复则不现实。本文提出QUASAR,一种在训练循环中持续进行轻量级、损失感知重构的方法,以降低损失下限并提升低比特模型性能。每步训练中,QUASAR利用平方梯度指数移动平均作为在线显著性估计,搜索小范围截断区间,并通过显著性加权最小二乘拟合仿射反量化器。分析表明,损失感知重构误差是QAT收敛界中唯一依赖重构的项,直接决定最终量化模型的损失,确立了该目标的理论合理性。QUASAR仅修改训练流程,支持标准部署格式(如整数量化、NVFP4),无推理时开销。在Qwen3与Llama-3.1上,2、3、4比特下均达到最低持留KL散度,3/4比特下KL降低≥10%,2比特下降低29%;2比特时在8个任务上平均准确率较强基线提升3.5-4.3个百分点。
原文摘要 · Abstract (English)
As large language model inference shifts toward lower precision, post-training quantization (PTQ) becomes increasingly brittle, making quantization-aware training (QAT) essential for preserving model quality. However, QAT computes the loss and surrogate gradients using a lossy reconstruction of latent full-precision weights, while applying updates to the latent weights themselves. This mismatch can lead to suboptimal training trajectories and a higher loss floor. Second-order PTQ methods mitigate a similar gap by minimizing loss-aware reconstruction error, but doing it once for a frozen model can take hours; repeating this process throughout QAT as the weights evolve is impractical. We introduce QUASAR, a QAT method that continuously performs lightweight, loss-aware reconstruction in the training loop to lower the loss floor and improve the resulting low-bit model. At each training step, QUASAR uses the exponential moving average of squared gradients as online saliency estimates, searches over a small set of clipping ranges, and fits affine dequantizers via saliency-weighted least squares. Our analysis shows that the loss-aware reconstruction error is the only reconstruction-dependent term in the QAT convergence bound and controls the loss of the final quantized model, establishing QUASAR's objective as a principled optimization target. QUASAR modifies only the training procedure and supports standard deployment formats, including integer quantization and NVFP4, with no inference-time changes or overhead. Across Qwen3 and Llama-3.1, QUASAR achieves the lowest held-out KL divergence among competitive QAT methods at 2, 3, and 4 bits, reducing KL by at least 10% at 3 and 4 bits and by 29% at 2 bits. At 2 bits, it improves average accuracy across eight tasks by 3.5-4.3 percentage points over strong QAT and PTQ baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。