提出AMXFP4，用非对称缩放解决4比特大模型推理中的激活异常问题。
AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inference
- 采用非对称共享缩放的微缩放浮点格式，无需校准直接量化
- 在VQA上比MXFP4高3%，在CSQA上比旋转法高1.6%
- 适合追求高效低精度推理的部署场景
随着大语言模型参数量和上下文长度的增长，计算精度已从16位降至4位以提升推理效率。然而，这种降低导致因激活异常值引发的准确率下降。基于旋转的INT4方法通过矩阵校准缓解此问题，但引入数小时开销且关键计算仍保留全精度。微缩放（MX）浮点格式通过共享缩放提供细粒度表示，支持完全量化矩阵乘法，无需校准。然而现有研究显示其在MXFP4推理中表现不佳，鲁棒性也未被充分探索。本文揭示了MX格式的根本权衡：虽能有效抑制激活异常值，却带来更大的组内不对称性。为此，我们提出AMXFP4——一种4位非对称浮点格式，通过非对称共享缩放同时解决两类问题，无需校准。定制的MAC引擎仅增加可忽略硬件开销，显著提升精度：在VQA上比MXFP4高3%，在CSQA上超过旋转法1.6%，并优于近期部署的商业版MXFP4变体。代码见：https://github.com/aiha-lab/MX-QLLM
原文摘要 · Abstract (English)
As large language models (LLMs) grow in parameter size and context length, computation precision has been reduced from 16-bit to 4-bit to improve inference efficiency. However, this reduction causes accuracy degradation due to activation outliers. Rotation-based INT4 methods address this via matrix calibration, but they introduce multi-hour overheads and leave key computations in full precision. Microscaling (MX) floating-point (FP) formats offer fine-grained representation with a shared scale, enabling fully quantized matrix multiplications through direct casting without calibration. However, existing research shows unsatisfactory empirical results for MXFP4 inference, and the robustness of MX formats remains largely unexplored. In this work, we uncover the fundamental tradeoffs of the MX format: while it effectively suppresses activation outliers, it does so at the cost of increased group-wise asymmetry. To address this, we propose AMXFP4, a 4-bit asymmetric FP format that handles both issues using asymmetric shared scales, without requiring calibration. Our custom MAC engine adds negligible hardware cost while improving accuracy: AMXFP4 outperforms MXFP4 by 3% on VQA and exceeds rotation-based methods by 1.6% on CSQA. It also surpasses recently deployed commercial MXFP4 variants. Code: https://github.com/aiha-lab/MX-QLLM
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。