arXiv:2604.17789cs.CVcs.AI2026-04被引 8

针对大模型推理中的量化误差问题,提出细粒度旋转优化方法。

DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization

  • 通过匹配微缩放块大小的细粒度旋转,精准定位异常值
  • 在LLaMA-3上实现比现有方法更优的性能,误差显著降低
  • 适合追求高效高精度推理的开发者和研究者

MXFP4微缩放格式将张量划分为32个元素共享一个E8M0缩放因子的块,已成为高效大模型推理的有前景基础,获得NVIDIA Blackwell Tensor Cores原生支持。然而,激活异常值在此格式下带来独特挑战:单个异常值会放大共享块的缩放因子,压缩其余元素的有效动态范围,导致严重量化误差。现有基于旋转的方法(如随机Hadamard和可学习旋转)为数据无关型,无法针对性地处理异常值集中通道。本文提出DuQuant++,将DuQuant的异常值感知细粒度旋转适配至MXFP4格式,使旋转块大小与微缩放组大小(B=32)对齐。由于每个MXFP4组拥有独立缩放因子,原始DuQuant中需双旋转与之字形排列的跨块方差问题不再存在,因此可仅用单一异常值感知旋转替代整个流程,旋转开销减半,同时平滑权重分布。在LLaMA-3系列模型上,采用MXFP4 W4A4量化进行的大量实验表明,DuQuant++持续达到业界最优性能。代码已开源:https://github.com/Hsu1023/DuQuant-v2。

原文摘要 · Abstract (English)

The MXFP4 microscaling format, which partitions tensors into blocks of 32 elements sharing an E8M0 scaling factor, has emerged as a promising substrate for efficient LLM inference, backed by native hardware support on NVIDIA Blackwell Tensor Cores. However, activation outliers pose a unique challenge under this format: a single outlier inflates the shared block scale, compressing the effective dynamic range of the remaining elements and causing significant quantization error. Existing rotation-based remedies, including randomized Hadamard and learnable rotations, are data-agnostic and therefore unable to specifically target the channels where outliers concentrate. We propose DuQuant++, which adapts the outlier-aware fine-grained rotation of DuQuant to the MXFP4 format by aligning the rotation block size with the microscaling group size (B{=}32). Because each MXFP4 group possesses an independent scaling factor, the cross-block variance issue that necessitates dual rotations and a zigzag permutation in the original DuQuant becomes irrelevant, enabling DuQuant++ to replace the entire pipeline with a single outlier-aware rotation, which halves the online rotation cost while simultaneously smoothing the weight distribution. Extensive experiments on the LLaMA-3 family under MXFP4 W4A4 quantization show that DuQuant++ consistently achieves state-of-the-art performance. Our code is available at https://github.com/Hsu1023/DuQuant-v2.

量化大模型推理微缩放旋转优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。