通过块尺度搜索提升大模型4比特量化精度，性能接近全精度。
ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initialization
- 设计块尺度搜索机制，自动选择最优量化尺度
- 在权重、激活等全链路量化下保持93%以上原始性能
- 理论推导缩放范围边界，效率远超传统方法
NVFP4是一种新型硬件支持的FP4格式，通过细粒度块尺度提升4比特量化精度。但现有方法仍主要依赖AbsMax初始化，与最优解存在明显差距。为此，本文提出ScaleSweep，一种简单高效的尺度优化方法：在可行块尺度候选集中搜索，选取使目标函数最小的候选值。我们进一步对NVFP4量化进行理论分析，推导出基于均方误差（MSE）和加权均方误差（WMSE）的上下界，显著缩小搜索空间，同时保证找到最优解，计算开销几乎可忽略。在Llama和Qwen模型上的实验表明，ScaleSweep持续优于现有初始化方法，进一步缩小与全精度模型的差距。尤其在权重、激活、KV缓存和查询状态的端到端量化中，性能保留超过93%。
原文摘要 · Abstract (English)
NVFP4 is a recently introduced hardware-supported FP4 format that improves the fidelity of 4-bit quantization through fine-grained block scales. However, existing NVFP4 scale initialization methods still primarily rely on AbsMax initialization, which leaves a noticeable gap to the optimal solution. To address this, we propose ScaleSweep, a simple and efficient scale optimization method that sweeps over feasible block scale candidates and selects the candidate that minimizes a target objective. We further provide a theoretical analysis of NVFP4 quantization and derive both lower and upper bounds for the required sweep range under mean square error (MSE) and weighted mean square error (WMSE) between the original tensor and the quantized reconstructed tensor. The proposed bounds substantially reduce the sweep space while preserving the optimal candidate, enabling negligible overhead compared with the baseline quantization operators. Experiments on Llama and Qwen models demonstrate that ScaleSweep consistently improves quantization performance over existing initialization methods and further narrows the gap to full precision. In particular, under aggressive end-to-end quantization of weights, activations, KV cache, and query states, ScaleSweep preserves more than 93% of the full-precision performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。