首次为VAE训练提供非渐近收敛保证，解释了优化器与超参影响。
Theoretical Convergence Guarantees for Variational Autoencoders
- 基于SGD和Adam算法，推导出收敛速率与迭代次数的关系。
- 理论证明收敛速度为O(log n / √n)，明确依赖批大小和变分采样数。
- 适用于线性VAE、深度高斯VAE及β-VAE、IWAE等变体，可指导调参。
变分自编码器（VAE）是广泛使用的生成模型，用于从复杂数据分布中采样。尽管在各类机器学习任务中表现出色，其理论性质仍存在显著空白，尤其缺乏收敛性保证。本文旨在填补这一空白，为使用随机梯度下降（SGD）和Adam算法训练的VAE提供非渐近收敛保证。我们推导出收敛速率为 $\mathcal{O}(\log n / \sqrt{n})$，其中 $n$ 为优化算法的迭代次数，并显式给出了对批大小、变分样本数量及其他关键超参数的依赖关系。该理论分析适用于线性VAE、深置高斯VAE以及β-VAE、IWAE等多种变体。此外，我们通过实验展示了超参数对收敛的影响，为理解VAE训练提供了新的理论洞见。
原文摘要 · Abstract (English)
Variational Autoencoders (VAE) are popular generative models used to sample from complex data distributions. Despite their empirical success in various machine learning tasks, significant gaps remain in understanding their theoretical properties, particularly regarding convergence guarantees. This paper aims to bridge that gap by providing non-asymptotic convergence guarantees for VAE trained using both Stochastic Gradient Descent and Adam algorithms. We derive a convergence rate of $\mathcal{O}(\log n / \sqrt{n})$, where $n$ is the number of iterations of the optimization algorithm, with explicit dependencies on the batch size, the number of variational samples, and other key hyperparameters. Our theoretical analysis applies to both Linear VAE and Deep Gaussian VAE, as well as several VAE variants, including $β$-VAE and IWAE. Additionally, we empirically illustrate the impact of hyperparameters on convergence, offering new insights into the theoretical understanding of VAE training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。