提出非对称正则化机制,让GAN训练更稳定、更快收敛。
Asymmetric regularization mechanism for GAN training with Variational Inequalities
- 基于变分不等式构建GAN训练的纳什均衡求解框架。
- 在光滑和局部可识别条件下,实现单次调用的线性收敛。
- 适合关注训练稳定性与收敛性的生成模型研究者。
我们将生成对抗网络(GAN)的训练问题建模为纳什均衡求解问题。为稳定训练过程并找到纳什均衡,提出一种基于经典Tikhonov正则化和新颖零中心梯度惩罚的非对称正则化机制。在光滑性和由高斯-牛顿格拉姆矩阵诱导的局部可识别性条件下,我们得到了正则化算子的显式利普希茨常数与(强)单调性常数。这些常数保证了单次调用的过去外推法(EFTP)具有逐次迭代线性收敛性。在学术案例上的实验表明,即使无法实现强单调性,该非对称正则化仍足以使训练收敛至均衡并稳定轨迹。
原文摘要 · Abstract (English)
We formulate the training of generative adversarial networks (GANs) as a Nash equilibrium seeking problem. To stabilize the training process and find a Nash equilibrium, we propose an asymmetric regularization mechanism based on the classic Tikhonov step and on a novel zero-centered gradient penalty. Under smoothness and a local identifiability condition induced by a Gauss-Newton Gramian, we obtain explicit Lipschitz and (strong)-monotonicity constants for the regularized operator. These constants ensure last-iterate linear convergence of a single-call Extrapolation-from-the-Past (EFTP) method. Empirical simulations on an academic example show that, even when strong monotonicity cannot be achieved, the asymmetric regularization is enough to converge to an equilibrium and stabilize the trajectory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。