一个模型支持多种精度推理，灵活适配不同硬件。
MF-QAT: Multi-Format Quantization-Aware Training for Elastic Inference
- 训练时兼容多种量化格式，提升模型泛化能力
- 单个模型在各精度下表现接近专用模型，无性能损失
- 支持运行时动态转换精度，适合多场景部署
量化感知训练（QAT）通常针对单一数值格式进行，但实际部署中需根据硬件或运行时约束选择精度。本文研究多格式量化感知训练，使单一模型在多种量化格式下均保持鲁棒性。实验表明，多格式QAT在每种目标精度下的性能可媲美单格式QAT，且对训练中未见的格式也表现良好。为实现实用部署，提出适用于MXINT和MXFP的切片与缩放转换方法，可在不重新训练的情况下将高精度表示转为低精度格式。构建了包含三步的流程：（i）使用多格式QAT训练模型；（ii）存储单一锚点格式检查点（MXINT8/MXFP8）；（iii）运行时按需转换至更低精度格式，准确率几乎无下降。该方案为弹性精度扩展提供可行路径，支持跨部署环境动态选择推理精度。
原文摘要 · Abstract (English)
Quantization-aware training (QAT) is typically performed for a single target numeric format, while practical deployments often need to choose numerical precision at inference time based on hardware support or runtime constraints. We study multi-format QAT, where a single model is trained to be robust across multiple quantization formats. We find that multi-format QAT can match single-format QAT at each target precision, yielding one model that performs well overall across different formats, even formats that were not seen during training. To enable practical deployment, we propose the Slice-and-Scale conversion procedure for both MXINT and MXFP that converts a high-precision representation into lower-precision formats without re-training. Building on this, we introduce a pipeline that (i) trains a model with multi-format QAT, (ii) stores a single anchor format checkpoint (MXINT8/MXFP8), and (iii) allows on-the-fly conversion to lower MXINT or MXFP formats at runtime with negligible-or no-additional accuracy degradation. Together, these components provide a practical path to elastic precision scaling and allow selecting the runtime format at inference time across diverse deployment targets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。