用EMA模型当老师指导生成器，提升图像质量和训练稳定性。
Improving Generative Adversarial Networks with Self-Distillation
- 用EMA生成器作教师，通过感知损失指导主生成器训练。
- 在多个数据集上降低FID和随机FID，改善图像质量。
- 适合需要稳定训练和高质量输出的生成模型研究者。
在现代生成对抗网络中，保持生成器权重的指数移动平均（EMA）是标准做法，因为该平均模型始终优于活跃训练的生成器。然而，EMA生成器仅用于最终部署，不参与训练过程。为解决这一机会缺失问题，我们提出自蒸馏生成对抗网络（SD-GAN），利用EMA生成器作为教师，通过感知损失指导活跃生成器（学生）。我们在狄拉克GAN设置下证明了SD-GAN的局部渐近稳定性，并表明其能抑制传统GAN中常见的寄生循环行为。在多个主流架构和数据集上的实证评估显示，SD-GAN在多个指标（尤其是FID和随机FID）上提升了最终图像质量，稳定了优化轨迹，并提供了与传统对抗损失无关的额外学习引导。该方法对微调预训练的GAN模型也有效。
原文摘要 · Abstract (English)
In modern GANs, maintaining an Exponential Moving Average (EMA) of the generator's weights is a standard practice, as such an averaged model consistently outperforms the actively trained generator. However, the EMA generator is used for final deployment only and does not influence the training process. To address this missed opportunity, we introduce Self-Distilled GAN (SD-GAN) that employs the EMA generator as a teacher to guide the active generator (student) via perceptual loss. We prove the local asymptotic stability of SD-GAN in the Dirac-GAN setting and show that it dampens the parasitic cycling behavior that plagues the conventional GANs. Empirical evaluations across established architectures and datasets demonstrate that SD-GAN improves the final image quality on several metrics (FID and random-FID in particular), stabilizes the optimization trajectory and provides additional learning guidance that is not trivially correlated with the conventional adversarial loss. It also proves effective for fine-tuning pretrained GAN models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。