通过拓扑感知的专家部署，显著降低MoE模型推理时的网络流量。
Cluster Topology-Driven Placement of Experts Reduces Network Traffic in MoE Inference
- 基于整数规划优化专家在多服务器间的分布
- 小规模（16B）和大规模（671B）模型均减少传输量
- 适合部署大规模MoE模型的系统工程师
将预训练的大语言模型高效部署到多服务器集群是快速响应用户查询的关键。近年来，混合专家（MoE）大模型的兴起带来了新的挑战：推理时仅部分专家被激活，且专家负载极不均衡。为提升集群利用率，需通过模型部署算法将模型分配到大量服务器上，而该算法必须考虑网络拓扑结构。本文提出一种整数线性规划（ILP）方法，用于确定专家最优部署位置，以最小化预期传输次数。由于模型内部结构特性，该优化问题可由标准ILP求解器高效求解。实验表明，该策略在小规模（DeepSeekMoE~16B）和大规模（DeepSeek-R1~671B）模型上均优于现有方法，有效降低网络流量。
原文摘要 · Abstract (English)
Efficient deployment of a pre-trained LLM to a cluster with multiple servers is a critical step for providing fast responses to users' queries. The recent success of Mixture-of-Experts (MoE) LLMs raises the question of how to deploy them efficiently, considering their underlying structure. During the inference in MoE LLMs, only a small part of the experts is selected to process a given token. Moreover, in practice, the experts' load is highly imbalanced. For efficient deployment, one has to distribute the model across a large number of servers using a model placement algorithm. Thus, to improve cluster utilization, the model placement algorithm has to take into account the network topology. This work focuses on the efficient topology-aware placement of the pre-trained MoE LLMs in the inference stage. We propose an integer linear program (ILP) that determines the optimal placement of experts, minimizing the expected number of transmissions. Due to the internal structure, this optimization problem can be solved with a standard ILP solver. We demonstrate that ILP-based placement strategy yields lower network traffic than competitors for small-scale (DeepSeekMoE~16B) and large-scale (DeepSeek-R1~671B) models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。