低成本无交换机网络在MoE大模型服务中更高效，可降本超50%。
Rethinking Network Topologies for Cost-Effective Mixture-of-Experts LLM Serving
- 对比四种GPU/TPU拓扑，无交换机结构成本更低
- 3D全连接拓扑在性能与成本间最优，提升56.2%
- 降低链路带宽可提升每成本吞吐量达27%
Mixture-of-experts (MoE) 架构使大语言模型（LLM）服务成为集群规模的任务，通信开销占运行时间相当大比例。这促使产业投入巨资于昂贵的高带宽扩展型网络。我们质疑此类高成本基础设施是否必要。本文首次系统性地跨层分析了MoE LLM服务中的网络成本效益，比较了四种典型XPU（如GPU/TPU）拓扑：扩展型、横向扩展型、3D环面和3D全连接。结果表明，在所有评估的服务场景中，低成本无交换机拓扑均优于扩展型拓扑，成本效益提升20.6%-56.2%。尤其，3D全连接拓扑在性能-成本权衡上为帕累托最优。此外，当前扩展型链路带宽存在过度配置：降低链路带宽可使每单位成本吞吐量提升最高达27%。对下一代GPU的前瞻性分析显示，无交换机网络的成本-性能优势将持续存在。
原文摘要 · Abstract (English)
Mixture-of-experts (MoE) architectures have turned LLM serving into a cluster-scale workload in which communication consumes a considerable portion of LLM serving runtime. This has prompted industry to invest heavily in expensive high-bandwidth scale-up networks. We question whether such costly infrastructure is strictly necessary. We present the first systematic cross-layer analysis of network cost-effectiveness for MoE LLM serving, comparing four representative XPU (e.g., GPU/TPU) topologies (scale-up, scale-out, 3D torus, and 3D full-mesh). We find that lower-cost switchless topologies are more cost-effective than the scale-up topology across all serving scenarios explored, improving cost-effectiveness by 20.6-56.2%. In particular, the 3D full-mesh topology is Pareto-optimal in terms of the performance-cost tradeoff. We also find that current scale-up link bandwidths are over-provisioned: reducing the link bandwidth improves throughput per cost by up to 27%. A forward-looking analysis of upcoming GPU generations indicates that the cost-performance advantage of switchless networks will likely persist.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。