动态切换量化，让MoE模型跨数据集高效运行
DynaMo: Runtime Switchable Quantization for MoE with Cross-Dataset Adaptation
- 按专家和通道分级设计可动态切换的量化策略
- 跨数据集测试中降低2.78~4.54点PPL，提升1.85%~3.77%准确率
- 适合需要多场景部署的大模型应用开发者
随着混合专家（MoE）架构参数量增加，模型量化需求愈发迫切。然而现有量化方法忽略了MoE在不同数据集上的专家动态特性，且静态量化难以适应数据变化。本文通过多层级分析揭示了MoE的动态规律，定义了各通道与专家的重要性。基于此，提出端到端的DynaMo量化框架：采用专家级混合精度基线量化策略，确保模型兼容多个现有数据集；并引入通道级动态切换机制，使量化后的MoE能自适应新数据集。实验表明，DynaMo在多个数据集上实现2.78~4.54的困惑度（PPL）下降，准确率提升1.85%~3.77%，推理速度提升约3倍，开销可忽略。
原文摘要 · Abstract (English)
As the Mix-of-Experts (MoE) architecture increases the number of parameters in large models, there is an even greater need for model quantization. However, existing quantization methods overlook the expert dynamics of MoE across multiple datasets. Moreover, the existing static quantization cannot adapt MoE to various data change scenarios. In this paper, we perform a multi-level analysis to reveal MoE dynamics and define the significance of each channel/each expert. Based on the analysis results, we propose \textit{DynaMo}, an end-to-end MoE quantization framework. DynaMo adopts an expert-level mixed-precision baseline quantization strategy, which ensures the quantized MoEs are compatible with multiple existing datasets. Furthermore, DynaMo incorporates a channel-level dynamic switching mechanism to adapt these quantized MoE models to novel datasets. Experiments show that DynaMo achieves a 2.78~4.54 PPL decrease and a 1.85%~3.77% accuracy improvement in various datasets, with ~3x inference speedup and negligible overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。