MxMoE通过混合精度量化提升MoE模型效率，兼顾准确率与速度。
MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design
- 根据敏感度和专家激活频率设计混合精度配置
- 2.25比特下比GPTQ低2.4点Wikitext-2困惑度
- 支持异构精度并行计算，适合部署优化场景
Mixture-of-Experts (MoE) 模型因参数量大、计算需求高而面临部署挑战。本文研究了MoE模型的量化问题，发现两个关键洞察：1）线性模块存在不同的量化敏感度；2）专家激活频率差异导致计算特征不均。基于此，提出MxMoE框架，从算法与系统双视角协同优化混合精度配置，综合考虑参数敏感度、专家激活动态与硬件资源，在设计空间中生成高效配置。同时，MxMoE自动生成优化的混合精度GroupGEMM核，支持不同精度GEMM的并行执行。实验表明，MxMoE在2.25比特下比GPTQ低2.4点Wikitext-2困惑度，全精度对比下最高提速3.4倍，5比特权重-激活量化下较统一量化提升29.4%速度且保持相同准确率。代码已开源：https://github.com/cat538/MxMoE。
原文摘要 · Abstract (English)
Mixture-of-Experts (MoE) models face deployment challenges due to their large parameter counts and computational demands. We explore quantization for MoE models and highlight two key insights: 1) linear blocks exhibit varying quantization sensitivity, and 2) divergent expert activation frequencies create heterogeneous computational characteristics. Based on these observations, we introduce MxMoE, a mixed-precision optimization framework for MoE models that considers both algorithmic and system perspectives. MxMoE navigates the design space defined by parameter sensitivity, expert activation dynamics, and hardware resources to derive efficient mixed-precision configurations. Additionally, MxMoE automatically generates optimized mixed-precision GroupGEMM kernels, enabling parallel execution of GEMMs with different precisions. Evaluations show that MxMoE outperforms existing methods, achieving 2.4 lower Wikitext-2 perplexity than GPTQ at 2.25-bit and delivering up to 3.4x speedup over full precision, as well as up to 29.4% speedup over uniform quantization at equivalent accuracy with 5-bit weight-activation quantization. Our code is available at https://github.com/cat538/MxMoE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。