系统评测大模型在微尺度浮点格式下的后训练量化效果
Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats
- 对比15个基准、3类大模型，测试7种量化算法在MXFP格式的表现
- MXFP8几乎无损，而MXFP4精度下降严重且难优化
- 发现量化敏感性主要由语言模型决定，适合做低精度部署的研究者必看
微尺度浮点（MXFP）已成为大语言模型（LLM）低精度计算的有前景格式。尽管已有多种后训练量化（PTQ）算法被提出，但它们大多聚焦于整数量化，对在MXFP格式下的适用性和行为仍缺乏系统研究。为此，本文系统评估了在MXFP格式下超过7种PTQ算法的表现，涵盖15个评估基准和3类大模型家族。关键发现包括：1）MXFP8可实现近乎无损性能，而MXFP4导致显著精度下降且优化困难；2）PTQ在MXFP下的有效性高度依赖格式兼容性，某些算法范式始终更优；3）量化表现趋势在不同模型家族与模态间高度一致，多模态大模型中量化敏感性主要由语言模型主导；4）量化缩放因子是MXFP4中的关键误差源，简单的预缩放优化策略可显著缓解其影响。这些结果为现有PTQ方法适配MXFP量化提供了实用指导。
原文摘要 · Abstract (English)
Microscaling Floating-Point (MXFP) has emerged as a promising low-precision format for large language models (LLMs). Despite various post-training quantization (PTQ) algorithms being proposed, they mostly focus on integer quantization, while their applicability and behavior under MXFP formats remain largely unexplored. To address this gap, this work conducts a systematic investigation of PTQ under MXFP formats, encompassing over 7 PTQ algorithms, 15 evaluation benchmarks, and 3 LLM families. The key findings include: 1) MXFP8 consistently achieves near-lossless performance, while MXFP4 introduces substantial accuracy degradation and remains challenging; 2) PTQ effectiveness under MXFP depends strongly on format compatibility, with some algorithmic paradigms being consistently more effective than others; 3) PTQ performance exhibits highly consistent trends across model families and modalities, in particular, quantization sensitivity is dominated by the language model rather than the vision encoder in multimodal LLMs; 4) The scaling factor of quantization is a critical error source in MXFP4, and a simple pre-scale optimization strategy can significantly mitigate its impact. Together, these results provide practical guidance on adapting existing PTQ methods to MXFP quantization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。