让GPU直接控制通信，提升超算与深度学习性能
GPU-centric Communication Schemes for HPC and ML Applications
- 将通信控制权从CPU转移到GPU，减少主机内存中转
- 利用GPU与网卡能力，实现零拷贝通信，降低延迟
- 适合大规模并行计算与深度学习场景的高性能系统
现代异构超级计算系统由CPU、GPU和高速网络互连（NIC）组成。并行化是高效利用这些系统执行可扩展仿真和深度学习任务的关键技术。分布式执行带来的进程间通信是性能瓶颈的重要因素。现有编程模型和运行时系统虽支持GPU感知通信，能直接将GPU上的通信缓冲区传至网卡而无需经过主机内存，但仍需CPU线程协调通信操作。本文综述了多种将通信控制路径从CPU转移至GPU的GPU-centric通信方案，分析其必要性、实现所需的GPU与NIC能力，以及潜在应用场景，并讨论了支持此类方案面临的技术挑战。
原文摘要 · Abstract (English)
Compute nodes on modern heterogeneous supercomputing systems comprise CPUs, GPUs, and high-speed network interconnects (NICs). Parallelization is identified as a technique for effectively utilizing these systems to execute scalable simulation and deep learning workloads. The resulting inter-process communication from the distributed execution of these parallel workloads is one of the key factors contributing to its performance bottleneck. Most programming models and runtime systems enabling the communication requirements on these systems support GPU-aware communication schemes that move the GPU-attached communication buffers in the application directly from the GPU to the NIC without staging through the host memory. A CPU thread is required to orchestrate the communication operations even with support for such GPU-awareness. This survey discusses various available GPU-centric communication schemes that move the control path of the communication operations from the CPU to the GPU. This work presents the need for the new communication schemes, various GPU and NIC capabilities required to implement the schemes, and the potential use-cases addressed. Based on these discussions, challenges involved in supporting the exhibited GPU-centric communication schemes are discussed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。