用专家路由提升量化模型精度，动态选最优量化方案
MoQE: Improve Quantization Model performance via Mixture of Quantization Experts
- 引入多专家架构，按输入特征动态选择最优量化路径
- 在多个模型上达到与顶尖量化方法相当的性能
- 轻量路由模块适配图像与文本任务，推理延迟几乎不变
量化技术对提升模型效率、降低部署成本至关重要，使深度学习模型能在资源受限设备上广泛应用。然而量化过程不可避免地导致精度下降。本文提出基于混合专家（MoE）架构的量化推理框架MoQE，将同一全精度模型的多种量化变体作为专业化的“量化专家”，根据输入数据特征动态路由至最合适的专家。通过专家化量化模型缓解单一定量化方案常见的性能退化问题。针对视觉与自然语言任务设计了轻量级、结构感知的路由模型。在ResNet、LLaMA和Qwen模型族上，于ImageNet、WikiText、C4和OpenWebText等基准数据集上的实验表明，MoQE在不显著增加推理延迟的情况下，性能可媲美当前最优量化方法。
原文摘要 · Abstract (English)
Quantization method plays a crucial role in improving model efficiency and reducing deployment costs, enabling the widespread application of deep learning models on resource-constrained devices. However, the quantization process inevitably introduces accuracy degradation. In this paper, we propose Mixture of Quantization Experts( abbr. MoQE), a quantization inference framework based on the Mixture-of-Experts (MoE) architecture, aiming to jointly improve the performance of quantization models. MoQE combines multiple quantization variants of one full-precision model as specialized "quantization experts" and dynamically routes input data to the most suitable expert based on its characteristics. MoQE alleviates the performance degradation commonly seen in single quantization models through specialization quantization expert models. We design lightweight, structure-aware router models tailored for both CV and NLP tasks. Experimental evaluations on ResNet, LLaMA, and Qwen model families across benchmark datasets including ImageNet, WikiText, C4, and OpenWebText demonstrate that MoQE achieves performance comparable to SOTA quantization model, without incurring significant increases in inference latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。