arXiv:2507.17417cs.LG2025-07被引 2

系统评估大模型量化方法,揭示关键组件与性能关系。

A Comprehensive Evaluation on Quantization Techniques for Large Language Models

  • 拆解量化为预处理与误差补偿两步,厘清技术路径
  • 精细粒度提升性能但增加存储,旋转策略对FP4效果有限
  • 首次公平对比MXFP4/NVFP4格式,发现精度与格式影响显著

针对大语言模型的后训练量化(PTQ)可大幅降低内存占用和计算开销。尽管多项研究报道突破性成果,但因评估条件不一,方法间难以横向比较。为此,我们系统梳理前沿方法,统一条件下开展全面评估。首先,将现有量化方法解耦为预量化变换与量化误差缓解两阶段:前者通过数据分布平滑减少异常值影响,后者通过补偿机制改善性能。其次,评估不同粒度与对称性设置的影响。第三,分析最新MXFP4与NVFP4数据格式表现。实验表明:优化的旋转缩放在预量化中表现最佳;结合低秩补偿的GPTQ偶可超越纯GPTQ;更细粒度提升性能但增加存储开销;缩放因子格式与精度显著影响FP4表现,而适用于INT4的旋转策略对MXFP4/NVFP4增益有限,提示需进一步研究。

原文摘要 · Abstract (English)

For large language models (LLMs), post-training quantization (PTQ) can significantly reduce memory footprint and computational overhead. Model quantization is rapidly evolving. Though many papers report breakthrough results, they are often evaluated under different settings because a method typically contains multiple components. Analyzing connections among existing methods is important for deeper understanding. To bridge these gaps, we conduct an extensive review of state-of-the-art methods and perform comprehensive evaluations under the same conditions for fair comparison. To our knowledge, such a fair and extensive investigation remains critically underexplored. To better understand connections, first, we decouple published quantization methods into two steps: pre-quantization transformation and quantization error mitigation. The former is a preprocessing step that reduces outlier impact by flattening the data distribution; the latter offsets quantization errors to improve performance. Second, we evaluate and analyze the impact of different settings, including granularity and symmetry. Third, we analyze and evaluate the latest MXFP4 and NVFP4 data formats and their performance. Our experiments first demonstrate that optimized rotation and scaling yield the best pre-quantization performance, and that combining low-rank compensation with GPTQ can occasionally outperform GPTQ alone for error mitigation. Second, finer granularity improves performance but increases storage overhead. Third, we find that scaling-factor format and precision greatly affect FP4 performance, and that rotation-based strategies effective for INT4 offer limited gains for MXFP4 and NVFP4, motivating further study.

大模型量化后训练量化精度优化数据格式

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。