提出新量化方法,让模型在低精度下仍能保持高性能。
BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization
- 通过块级变换避免异常值跨块传播,提升稳定性。
- 在W4A4KV16配置下恢复96.43%的全精度性能。
- 适合部署多模态大模型和语言模型的高效率量化场景。
微缩浮点(MXFP)格式已成为在现代加速器上部署多模态大语言模型(MLLMs)和大语言模型(LLMs)的有前景标准。然而,现有后训练量化(PTQ)方法,尤其是针对整数格式设计的旋转技术,在应用于MXFP4时会出现严重性能下降。近期研究指出,这一失败源于根本性的格式不匹配:全局正交旋转会无意中将异常值能量传递至不同量化块,引发新异常值并破坏局部块级缩放,同时常导致双峰激活分布,未能充分利用有限的量化范围。为此,我们提出BATQuant(块级仿射变换),将变换限制在与MXFP粒度一致的范围内,防止跨块异常值传播,同时放宽正交性约束以优化分布形态。为确保参数效率,引入全局与私有克罗内克(GPK)分解,有效降低存储和运行开销,并结合块级可学习裁剪抑制残余异常值。在多模态和语言模型上的大量实验表明,BATQuant在极端压缩的W4A4KV16配置下达到新基准,于多模态基准上恢复高达96.43%的全精度性能,且在各类任务中显著优于现有方法。
原文摘要 · Abstract (English)
Microscaling floating-point (MXFP) formats have emerged as a promising standard for deploying Multi-modal Large Language Models (MLLMs) and Large Language Models (LLMs) on modern accelerator architectures. However, existing Post-Training Quantization (PTQ) methods, particularly rotation-based techniques designed for integer formats, suffer from severe performance collapse when applied to MXFP4. Recent studies attribute this failure to a fundamental format mismatch: global orthogonal rotations inadvertently transfer outlier energy across quantization blocks, inducing new outliers that disrupt local block-wise scaling, while often creating bimodal activation distributions that underutilize the limited quantization range. To address these issues, we propose BATQuant (Block-wise Affine Transformation), which restricts transformations to align with MXFP granularity to prevent cross-block outlier propagation, while relaxing orthogonality constraints to optimize distribution shaping. To ensure parameter efficiency, we introduce Global and Private Kronecker (GPK) decomposition to effectively reduces storage and runtime overhead and incorporate Block-wise Learnable Clipping to suppress residual outliers. Extensive experiments on both MLLMs and LLMs demonstrate that BATQuant establishes new state-of-the-art results under aggressive W4A4KV16 configurations, recovering up to 96.43% of full-precision performance on multimodal benchmarks and clearly outperforming existing methods across diverse tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。