提升MXFP4量化精度，让低功耗推理接近顶级模型表现
Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction
- 提出两种纯软件优化：动态范围扩展与粗粒度高精度缩放
- 将MXFP4与NVFP4的性能差距从10%降至1%以下
- 无需硬件改动，仅增加6.2%计算开销，适合部署在边缘设备
大语言模型对低精度格式的需求日益增长，以实现高效的大规模推理。开放计算项目（OCP）微缩标准（MX）因其出色的硬件效率而备受关注，但其4位变体（MXFP4）在精度上仍落后于NVIDIA的NVFP4，限制了应用。本文提出两种纯软件技术：溢出感知缩放（OAS）和宏块缩放（MBS），在不改变硬件的前提下提升MXFP4的量化保真度。OAS通过幂次方块缩放增强有效动态范围，降低整体误差；MBS在粗粒度层级分配更高精度缩放，更好保留异常值。在多个LLM和标准下游基准测试中，该方法使MXFP4与NVFP4的端到端精度差距从约10%降至平均低于1%，同时仅带来6.2%的平均GEMM开销。结果重新确立了MXFP4作为NVFP4的实用替代方案，在保持硬件效率优势（如张量核心面积节省12%）的同时实现接近NVFP4的精度。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have intensified the need for low-precision formats that enable efficient, large-scale inference. The Open Compute Project (OCP) Microscaling (MX) standard is attractive due to its favorable hardware efficiency, but its 4-bit variant (MXFP4) lags behind NVIDIA's NVFP4 in accuracy, limiting adoption. We introduce two software-only techniques, Overflow-Aware Scaling (OAS) and Macro Block Scaling (MBS), that improve MXFP4 quantization fidelity without requiring hardware changes. OAS reduces overall errors by increasing effective dynamic range under power-of-two block scaling, while MBS allocates higher-precision scaling at a coarser granularity to better preserve outliers. Across multiple LLMs and standard downstream benchmarks, OAS and MBS reduce the end-to-end accuracy gap between MXFP4 and NVFP4 from about 10% to below 1% on average, while incurring modest GEMM overhead (6.2% on average). These results re-establish MXFP4 as a practical alternative to NVFP4, enabling near-NVFP4 accuracy while retaining MX's hardware-efficiency advantages (e.g., 12% relative area savings in tensor cores).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。