新通信系统让专家模型在不同硬件上都能高效运行。
UCCL-EP: Portable Expert-Parallel Communication
- 用CPU代理发起远程直接内存访问,摆脱对GPU-NIC紧耦合的依赖。
- 在AWS EFA等不支持特定顺序语义的网卡上仍能正确运行,性能提升最高达2.1倍。
- 适用于NVIDIA/AMD多平台,适合需要跨硬件部署的大型模型训练场景。
混合专家(MoE)工作负载依赖专家并行(EP)实现高GPU利用率。现有先进EP通信系统如DeepEP虽性能优异,但在异构GPU与网卡平台间可移植性差。其根源在于:基于GPU发起的令牌级RDMA通信需紧密集成GPU与网卡,例如直接写入网卡驱动或MMIO接口。本文提出UCCL-EP,一种可移植的EP通信系统,在异构硬件上达到DeepEP级别的性能。UCCL-EP将GPU主导的RDMA替换为高吞吐的GPU-CPU控制通道:紧凑的令牌路由指令由多线程CPU代理接收,并代为发起GPUDirect RDMA操作。同时,通过使用RDMA立即数据模拟各类所需顺序语义,使系统在缺乏相应顺序保证的网卡(如AWS EFA)上仍能正确运行。我们在NVIDIA和AMD GPU上实现了UCCL-EP,搭配EFA与Broadcom网卡。在EFA平台上,其调度与合并吞吐量比现有最佳方案最高提升2.1倍;在纯NVIDIA平台,性能与原版DeepEP相当。此外,UCCL-EP在NVIDIA+EFA平台上使SGLang的令牌吞吐量提升最高达40%,在16节点的AMD+Broadcom平台下,使DeepSeek-V3训练吞吐量较AMD Primus/Megatron-LM框架最高提升45%。
原文摘要 · Abstract (English)
Mixture-of-Experts (MoE) workloads rely on expert parallelism (EP) to achieve high GPU efficiency. State-of-the-art EP communication systems such as DeepEP demonstrate strong performance but exhibit poor portability across heterogeneous GPU and NIC platforms. The poor portability is rooted in architecture: GPU-initiated token-level RDMA communication requires tight vertical integration between GPUs and NICs, e.g., GPU writes to NIC driver/MMIO interfaces. We present UCCL-EP, a portable EP communication system that delivers DeepEP-level performance across heterogeneous GPU and NIC hardware. UCCL-EP replaces GPU-initiated RDMA with a high-throughput GPU-CPU control channel: compact token-routing commands are transferred to multithreaded CPU proxies, which then issue GPUDirect RDMA operations on behalf of GPUs. UCCL-EP further emulates various ordering semantics required by specialized EP communication modes using RDMA immediate data, enabling correctness on NICs that lack such ordering, e.g., AWS EFA. We implement UCCL-EP on NVIDIA and AMD GPUs with EFA and Broadcom NICs. On EFA, it outperforms the best existing EP solution by up to $2.1\times$ for dispatch and combine throughput. On NVIDIA-only platform, UCCL-EP achieves comparable performance to the original DeepEP. UCCL-EP also improves token throughput on SGLang by up to 40% on the NVIDIA+EFA platform, and improves DeepSeek-V3 training throughput over the AMD Primus/Megatron-LM framework by up to 45% on a 16-node AMD+Broadcom platform.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。