提出非高斯数据下WGAN的最优解,实现线性复杂度高效生成。
On optimal solutions of classical and sliced Wasserstein GANs with non-Gaussian data
- 非线性激活+非高斯数据时,一维WGAN有闭式最优解
- 高维数据中线性生成器渐近最优,复杂度仅为线性
- 新方法比r-PCA快三倍以上,适合大规模数据生成
生成对抗网络(GAN)旨在通过参数化神经网络逼近未知分布。尽管广泛应用于强化学习、半监督学习和计算机视觉任务,其参数选择常需大量试错,且仅有少数方法被证明理论上最优。其中最具前景的变体是沃尔沙斯坦生成对抗网络(WGAN)。以往关于总体WGAN最优参数的研究仅限于线性-二次-高斯(LQG)场景,即生成器为线性且数据服从高斯分布。本文研究超越LQG设置的总体WGAN最优解。首先,当生成器具有非线性激活函数且数据非高斯时,推导出一维WGAN的闭式最优参数。对于高维数据,采用切片沃尔沙斯坦框架,证明线性生成器可渐近最优。此外,传统切片WGAN仅约束投影数据边缘分布,而未约束整体分布,因此我们提出新的无投影切片WGAN并确认其渐近最优性。实验表明,相较于具有立方复杂度的著名r-主成分分析(r-PCA)解法,我们的切片WGAN生成器仅需线性复杂度即可获得更优性能。
原文摘要 · Abstract (English)
The generative adversarial network (GAN) aims to approximate an unknown distribution via a parameterized neural network (NN). While GANs have been widely applied in reinforcement and semi-supervised learning as well as computer vision tasks, selecting their parameters often needs an exhaustive search, and only a few selection methods have been proven to be theoretically optimal. One of the most promising GAN variants is the Wasserstein GAN (WGAN). Prior work on optimal parameters for population WGAN is limited to the linear-quadratic-Gaussian (LQG) setting, where the generator NN is linear, and the data is Gaussian. In this paper, we focus on the characterization of optimal solutions of population WGAN beyond the LQG setting. As a basic result, closed-form optimal parameters for one-dimensional WGAN are derived when the NN has non-linear activation functions, and the data is non-Gaussian. For high-dimensional data, we adopt the sliced Wasserstein framework and show that the linear generator can be asymptotically optimal. Moreover, the original sliced WGAN only constrains the projected data marginal instead of the whole one in classical WGAN, and thus, we propose another new unprojected sliced WGAN and identify its asymptotic optimality. Empirical studies show that compared to the celebrated r-principal component analysis (r-PCA) solution, which has cubic complexity to the data dimension, our generator for sliced WGAN can achieve better performance with only linear complexity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。