让边缘设备高效运行MoE模型，不丢语义还提速超70%
ZipMoE: Efficient On-Device MoE Serving via Lossless Compression and Cache-Affinity Scheduling
- 通过缓存调度协同设计压缩参数，实现无损推理
- 在边缘设备上降低72.77%延迟，吞吐提升6.76倍
- 适合资源受限场景下需保持模型原貌的部署
尽管混合专家（MoE）架构显著提升了大语言模型的表达能力，但其巨大的内存开销严重阻碍了在资源受限的边缘设备上的实际部署，尤其当模型行为必须保持无损、不依赖有损量化时。本文提出ZipMoE，一种高效且语义无损的边缘MoE服务系统。该系统利用边缘硬件特性与MoE参数固有的统计冗余性之间的协同效应，采用具备理论性能保障的缓存-调度协同设计。其核心在于将边缘MoE推理从原本的I/O瓶颈转变为以计算为中心的工作流，支持高效并行化。我们实现了ZipMoE原型，并在代表性边缘平台使用主流开源MoE模型和真实工作负载进行了大量实验。评估结果显示，相比现有最先进系统，ZipMoE在推理延迟上最高降低72.77%，吞吐量最高提升6.76倍。代码已公开于：https://github.com/npnothard/ZipMoE-ICML26。
原文摘要 · Abstract (English)
While Mixture-of-Experts (MoE) architectures substantially bolster the expressive power of large-language models, their prohibitive memory footprint severely impedes the practical deployment on resource-constrained edge devices, especially when model behavior must be preserved without relying on lossy quantization. In this paper, we present ZipMoE, an efficient and semantically lossless on-device MoE serving system. ZipMoE exploits the synergy between the hardware properties of edge devices and the statistical redundancy inherent to MoE parameters via a caching-scheduling co-design with provable performance guarantee. Fundamentally, our design shifts the paradigm of on-device MoE inference from an I/O-bound bottleneck to a compute-centric workflow that enables efficient parallelization. We implement a prototype of ZipMoE and conduct extensive experiments on representative edge computing platforms using popular open-source MoE models and real-world workloads. Our evaluation reveals that ZipMoE achieves up to $72.77\%$ inference latency reduction and up to $6.76\times$ higher throughput than the state-of-the-art systems.Our code is available at: https://github.com/npnothard/ZipMoE-ICML26.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。