arXiv:2510.19366cs.CLcs.LG2025-10被引 2

让MoE模型按需激活专家,实现请求级算力弹性调度。

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs

  • 将大专家拆成细粒度子专家,支持更灵活的路由选择。
  • 路由操作点数量提升4倍,离线推理吞吐最高增33.9%。
  • 适合需要动态分配算力的高并发大模型服务场景。

混合专家(MoE)通过稀疏激活扩展模型容量,已成为大语言模型的重要架构。然而现有MoE服务系统通常使用固定路由配置处理所有请求,难以利用请求间的异构计算需求。路由top-$k$(每标记激活的专家数)直接控制被调用专家的计算量,是实现请求级算力弹性的天然机制。但其实现需更细粒度的路由单元和高效的运行时执行能力。本文提出 extsc{MoE-Prism},一个支持请求级算力弹性的模型与系统协同设计框架。该框架将单体专家分解为细粒度子专家,暴露更多路由操作点,并提供感知$k$值的运行时系统,在吞吐优先与低延迟负载下均能高效服务异构路由预算。我们在vLLM上实现 extsc{MoE-Prism},并在三个代表性MoE模型上评估。结果表明,其路由操作点数量扩大4倍,离线推理吞吐最高提升33.9%,在异构负载下显著降低在线服务的TTFT。这验证了请求级路由目标下实用的弹性MoE服务可行性。

原文摘要 · Abstract (English)

Mixture-of-Experts (MoE) scales model capacity through sparse activation, and is becoming an important architecture for large language models (LLMs). However, existing MoE serving systems typically execute all requests under a fixed routing configuration, limiting their ability to exploit heterogeneous computation requirements across requests. Routing top-$k$, which determines the number of routed experts activated per token, directly controls routed-expert computation and provides a natural mechanism for request-level compute elasticity. Realizing this capability, however, requires finer-grained routing units and efficient runtime execution for heterogeneous routing budgets. We present \textsc{MoE-Prism}, a model and system support framework for request-level compute elasticity in MoE serving. \textsc{MoE-Prism}decomposes monolithic experts into fine-grained sub-experts to expose denser routing operating points and provides a $k$-aware serving runtime that effectively serves heterogeneous routing budgets under both throughput-oriented and latency-sensitive workloads. We implement \textsc{MoE-Prism} on top of vLLM and evaluate it on three representative MoE models. \textsc{MoE-Prism} expands the number of available routing operating points by $4\times$, improves offline inference throughput by up to 33.9\%, and reduces online serving TTFT under heterogeneous workloads. These results demonstrate practical elastic MoE serving with request-level routing targets.

MoE弹性推理大模型服务

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。