Iris让多GPU编程既高效又简单，用Python和Triton实现计算通信无缝重叠。
Iris: First-Class Multi-GPU Programming Experience in Triton
- 基于分块对称内存，支持在单个内核中自然融合计算与通信。
- 微基准测试接近最优带宽利用率，GEMM+All-Scatter任务提速1.79倍。
- 适合希望简化多GPU编程且不牺牲性能的开发者使用。
多GPU编程传统上需在性能与可编程性间权衡：高性能实现依赖低层HIP/CUDA通信库，工程量大；而简单抽象常以性能为代价。我们提出Iris，一个完全用Python和Triton实现的多GPU通信库，彻底消除这一权衡。Iris提供基于分块的对称内存抽象，天然契合Triton编程模型，使开发者能编写单一源码内核，无缝交织计算与通信。我们提出一个计算-通信重叠模式分类法——从批量同步到细粒度工作组专用——在Iris中仅需少量代码修改即可实现，通常只需在相同内核中添加几行代码。评估显示，Iris在微基准测试中实现接近最优的带宽利用率，并在GEMM+All-Scatter工作负载上相较PyTorch和RCCL最高提速1.79倍，证明高层实现可媲美甚至超越高度优化的底层库，同时大幅简化多GPU编程。
原文摘要 · Abstract (English)
Multi-GPU programming traditionally requires developers to navigate complex trade-offs between performance and programmability. High-performance implementations typically rely on low-level HIP/CUDA communication libraries that demand substantial engineering effort for even basic overlap patterns, while simpler abstractions often sacrifice performance. We present Iris, a multi-GPU communication library implemented entirely in Python and Triton that eliminates this trade-off. Iris provides tile-based symmetric memory abstractions that naturally align with Triton's programming model, enabling developers to write single-source kernels that seamlessly interleave computation and communication. We demonstrate a taxonomy of compute-communication overlap patterns--from bulk-synchronous to fine-grained workgroup specialization--that can be implemented with minimal code changes in Iris, often requiring just a few additional lines within the same Triton kernel. Our evaluation shows that Iris achieves near-optimal bandwidth utilization in microbenchmarks and delivers up to 1.79x speedup over PyTorch and RCCL for GEMM+All-Scatter workloads, demonstrating that high-level implementations can match or exceed heavily-optimized libraries while dramatically simplifying multi-GPU programming.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。