ParaGAN加速GAN训练，15天缩短至14小时，提升30%吞吐量。
ParaGAN: A Scalable Distributed Training Framework for Generative Adversarial Networks
- 异步训练+非对称优化，配合智能数据管道与硬件布局转换。
- BigGAN训练从15天缩至14小时，吞吐量提升超30%，缩放效率达91%。
- 支持高分辨率图像生成，适合大规模生成模型研发者使用。
生成式人工智能的进展推动了众多应用，尤其是生成对抗网络（GAN）在合成真实照片和视频方面至关重要。然而，由于计算密集且数值不稳定，高效训练GAN仍面临挑战，现有方法常需数天甚至数周。本文提出ParaGAN，一种可扩展的分布式GAN训练框架，采用异步训练和非对称优化策略以加速训练。ParaGAN结合拥塞感知数据流水线与硬件感知布局变换，提升加速器利用率，使吞吐量提升超过30%。使用ParaGAN，BigGAN训练时间从15天缩短至14小时，实现91%的缩放效率。此外，该框架还支持前所未有的高分辨率图像生成。
原文摘要 · Abstract (English)
Recent advances in Generative Artificial Intelligence have fueled numerous applications, particularly those involving Generative Adversarial Networks (GANs), which are essential for synthesizing realistic photos and videos. However, efficiently training GANs remains a critical challenge due to their computationally intensive and numerically unstable nature. Existing methods often require days or even weeks for training, posing significant resource and time constraints. In this work, we introduce ParaGAN, a scalable distributed GAN training framework that leverages asynchronous training and an asymmetric optimization policy to accelerate GAN training. ParaGAN employs a congestion-aware data pipeline and hardware-aware layout transformation to enhance accelerator utilization, resulting in over 30% improvements in throughput. With ParaGAN, we reduce the training time of BigGAN from 15 days to 14 hours while achieving 91% scaling efficiency. Additionally, ParaGAN enables unprecedented high-resolution image generation using BigGAN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。