从神经正切核视角证明了过参数化VAE的收敛性，填补理论空白。
On the Convergence Analysis of Over-Parameterized Variational Autoencoders: A Neural Tangent Kernel Perspective
- 用神经正切核分析VAE中随机神经网络的优化轨迹
- 在温和假设下严格证明了过参数化VAE的收敛性
- 揭示了VAE优化与核岭回归的新关联，适合理论研究者
变分自编码器（VAEs）已成为生成任务中强大的概率模型，但其收敛性质尚未得到严格证明。训练目标的高度非凸性以及在VAE架构中实现的随机神经网络（SNN）使得收敛性分析极为困难。本文通过神经正切核（NTK）技术，刻画了用于VAE的SNN的优化轨迹，该技术能支配超宽神经网络的优化与泛化行为。我们在温和假设下提供了VAE收敛性的数学证明，从而推进了对VAE优化动态的理论理解。此外，我们建立了过参数化SNN所面临的优化问题与核岭回归（KRR）问题之间的新联系。研究结果不仅强化了VAE的理论基础，还为利用先进核方法研究生成模型优化开辟了新路径。理论结论通过实验模拟得到验证。
原文摘要 · Abstract (English)
Variational Auto-Encoders (VAEs) have emerged as powerful probabilistic models for generative tasks. However, their convergence properties have not been rigorously proven. The challenge of proving convergence is inherently difficult due to the highly non-convex nature of the training objective and the implementation of a Stochastic Neural Network (SNN) within VAE architectures. This paper addresses these challenges by characterizing the optimization trajectory of SNNs utilized in VAEs through the lens of Neural Tangent Kernel (NTK) techniques. These techniques govern the optimization and generalization behaviors of ultra-wide neural networks. We provide a mathematical proof of VAE convergence under mild assumptions, thus advancing the theoretical understanding of VAE optimization dynamics. Furthermore, we establish a novel connection between the optimization problem faced by over-parameterized SNNs and the Kernel Ridge Regression (KRR) problem. Our findings not only contribute to the theoretical foundation of VAEs but also open new avenues for investigating the optimization of generative models using advanced kernel methods. Our theoretical claims are verified by experimental simulations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。