用瑞尼交叉熵改进GAN，让训练更快更稳。
$α$-GAN by Rényi Cross Entropy
- 用瑞尼交叉熵构建新价值函数，以软判断的确定性为优化目标。
- 当α∈(0,1)时梯度呈指数放大，实验验证收敛速度显著提升。
- 适合解决梯度消失问题，尤其适用于训练困难的生成模型。
本文提出α-GAN，一种基于瑞尼测度的生成对抗网络。价值函数通过瑞尼交叉熵定义，衡量判别器对样本来源（真实分布或生成器）软决策的期望确定性。判别器试图最大化该确定性，而生成器则通过注入假样本降低它，形成由瑞尼阶α参数化的极小极大问题。当α=1时，α-GAN退化为经典GAN，此时价值函数即二元交叉熵。α-GAN的优化在概率向量空间中进行，发现当α∈(0,1)时梯度呈指数放大，加速了收敛，实验结果验证了这一点。讨论表明，选择α∈(0,1)可能缓解常见问题如梯度消失。进一步观察指出，现有瑞尼型GAN尚未充分探索该区间。
原文摘要 · Abstract (English)
This paper proposes $α$-GAN, a generative adversarial network using Rényi measures. The value function is formulated, by Rényi cross entropy, as an expected certainty measure incurred by the discriminator's soft decision as to where the sample is from, true population or the generator. The discriminator tries to maximize the Rényi certainty about sample source, while the generator wants to reduce it by injecting fake samples. This forms a min-max problem with the solution parameterized by the Rényi order $α$. This $α$-GAN reduces to vanilla GAN at $α= 1$, where the value function is exactly the binary cross entropy. The optimization of $α$-GAN is over probability (vector) space. It is shown that the gradient is exponentially enlarged when Rényi order is in the range $α\in (0,1)$. This makes convergence faster, which is verified by experimental results. A discussion shows that choosing $α\in (0,1)$ may be able to solve some common problems, e.g., vanishing gradient. A following observation reveals that this range has not been fully explored in the existing Rényi version GANs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。