arXiv:2607.24377cs.LGcs.AI2026-07

提出无需数据的MXFP4注意力量化方法,显著提升视频生成效率与质量。

MXAttention: Data-Free Optimal Scaling and Pre-Normalization Quantization for MXFP4 Attention

论文配图:MXAttention: Data-Free Optimal Scaling and Pre-Normalization Quantization for MXFP4 Attention
图 1 · 摘自论文原文
  • 通过周期性缩放与预归一化量化,解决数值误差问题。
  • 在两个视频模型上实现95%以上质量差距修复,仅0.01以下退化。
  • 适合追求高效推理的扩散模型部署者使用。

基于扩散的视频生成模型中,注意力机制的二次计算开销是主要瓶颈。MXFP4注意力为高效推理提供了可能,但直接量化常因两大数值问题导致生成质量下降:幂次缩放引发的截断-下溢权衡,以及softmax循环中行归一化引入的误差。本文提出MXAttention,一种无需数据的后训练量化框架。其包含两个组件:通用最优缩放(UOS),利用幂次微缩放的周期结构,推导出无需校准或搜索的分布无关最优缩放边界Qmax=7.25;预归一化量化(PNQ),在行求和前对未归一化的softmax指数进行量化,从而构造性地保持归一化。在Wan2.2与HunyuanVideo上的实验表明,MXAttention至少修复了OCP MXFP4与FP16之间95%的VBench图像质量差距,显著提升帧级相似性,并在所有报告的VBench指标上保持接近FP16水平的质量(绝对退化小于0.01)。该方法在融合进注意力流水线后,性能可媲美强基线NVFP4方案,且开销极小。代码已公开于MindIE-SD。

原文摘要 · Abstract (English)

The quadratic cost of attention is a major bottleneck in diffusion-based video generation models. MXFP4 attention provides a promising path toward efficient inference, but direct MXFP4 quantization often degrades generation quality due to two numerical issues: the clipping-underflow trade-off from power-of-two scaling and the row-wise normalization error introduced in the softmax loop. We propose MXAttention, a data-free post-training quantization framework for MXFP4 attention. MXAttention introduces two components: Universal Optimal Scaling (UOS), which exploits the periodic structure of power-of-two microscaling to derive a distribution-independent optimal scaling boundary Qmax=7.25 without calibration or search, and Pre-Normalization Quantization (PNQ), which quantizes unnormalized softmax exponentials before row-wise summation to preserve normalization by construction. Experiments on Wan2.2 and HunyuanVideo show that MXAttention closes at least 95% of the VBench Imaging Quality gap between OCP MXFP4 and FP16, substantially improves frame-level similarity, and preserves FP16-level generation quality with less than 0.01 absolute degradation on all reported VBench metrics. MXAttention also achieves performance competitive with strong NVFP4-based baselines with negligible overhead when fused into the attention pipeline. The implementation is publicly available in MindIE-SD.

量化注意力机制视频生成MXFP4

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。