通过细粒度调度提升稀疏专家模型的推理效率，显著加快大模型响应速度。
Efficient MoE Inference with Fine-Grained Scheduling of Disaggregated Expert Parallelism
- 将计算与通信拆分为更小任务，实现更精细的流水线并行。
- 在32张GPU上实测推理吞吐最高提升1.61倍，最快达1.24倍加速。
- 适合需要高并发、低延迟的大模型部署场景，如云端AI服务。
混合专家（MoE）架构可在计算量近似线性增长的情况下扩展模型规模，但其推理阶段因需存储键值缓存（KV caches）和稀疏激活专家而面临内存压力。近期提出的非集中式专家并行（DEP）将注意力计算与专家处理分配到专用GPU组，但缺乏对共享专家的支持及高效的任务调度机制，限制了性能表现。本文提出FinDEP，一种面向DEP的细粒度任务调度算法，通过最大化任务重叠来提升MoE推理吞吐量。该方法引入三项创新：1）将计算/通信分解为更小任务以支持细粒度流水线；2）构建可支持可变粒度与顺序的调度优化模型；3）设计高效求解器应对大规模搜索空间。在四套GPU系统上使用DeepSeek-V2与Qwen3-MoE进行实验，结果表明FinDEP相较先前方法最高提升1.61倍吞吐量，在32张GPU系统上实现最高1.24倍加速。
原文摘要 · Abstract (English)
The mixture-of-experts (MoE) architecture scales model size with sublinear computational increase but suffers from memory-intensive inference due to KV caches and sparse expert activation. Recent disaggregated expert parallelism (DEP) distributes attention and experts to dedicated GPU groups but lacks support for shared experts and efficient task scheduling, limiting performance. We propose FinDEP, a fine-grained task scheduling algorithm for DEP that maximizes task overlap to improve MoE inference throughput. FinDEP introduces three innovations: 1) partitioning computation/communication into smaller tasks for fine-grained pipelining, 2) formulating a scheduling optimization supporting variable granularity and ordering, and 3) developing an efficient solver for this large search space. Experiments on four GPU systems with DeepSeek-V2 and Qwen3-MoE show FinDEP improves throughput by up to 1.61x over prior methods, achieving up to 1.24x speedup on a 32-GPU system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。