arXiv:2511.04214cs.LGcs.CL2025-11被引 16

提出块旋转策略,让现有量化方法适配新型FP4格式

Block Rotation is All You Need for MXFP4 Quantization

  • 用块级旋转替代全局旋转,解决与MXFP4的兼容性问题
  • 在多个LLM上实现显著精度提升,最高达3.2个百分点
  • 为低精度部署提供新思路,适合模型压缩实践者

大语言模型规模迅速增长,带来高昂的内存、计算和能耗成本。后训练量化(PTQ)是高效部署的可行方案,但实现高精度的W4A4量化仍是挑战。现有方法多针对INT4设计,而新兴的MXFP4格式(获NVIDIA、AMD、Intel等支持)对现有技术提出新问题。本文建立首个基于MXFP4的PTQ基准测试,发现如GPTQ表现优异,而主流的旋转类方法因与MXFP4的幂次对齐机制不兼容,导致性能严重下降。我们深入分析指出,根源在于MXFP4的块级幂次缩放与全局旋转引发的异常能量分布之间的冲突。据此提出简单有效的块旋转策略,可使旋转类方法在多种LLM上实现显著精度提升,最高达3.2个百分点。研究成果为实践者提供明确指导,并推动面向新兴低精度格式的量化研究。

原文摘要 · Abstract (English)

Large language models (LLMs) have achieved remarkable success, but their rapidly growing scale imposes prohibitive costs in memory, computation, and energy. Post-training quantization (PTQ) is a promising solution for efficient deployment, yet achieving accurate W4A4 quantization remains an open challenge. While most existing methods are designed for INT4 formats, the emergence of MXFP4 -- a new FP4 format with various hardware support (NVIDIA, AMD, Intel)-- raises questions about the applicability of current techniques. In this work, we establish a comprehensive benchmark of PTQ methods under the MXFP4 format. Through systematic evaluation, we find that methods like GPTQ consistently deliver strong performance, whereas rotation-based approaches, which are almost used by all state-of-the-art approaches, suffer from severe incompatibility with MXFP4. We further provide the first in-depth analysis of this conflict, tracing its root to a fundamental mismatch between MXFP4's PoT (power-of-two) block scaling and the redistribution of outlier energy via global rotation. Building on this insight, we propose a simple yet effective block rotation strategy that adapts rotation-based methods to MXFP4, leading to substantial accuracy improvements across diverse LLMs. Our findings not only offer clear guidance for practitioners but also set a foundation for advancing PTQ research under emerging low-precision formats.

量化LLMMXFP4块旋转

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。