教你怎么用离散潜变量训练VAE，适合文本等数据类型。
An Introduction to Discrete Variational Autoencoders
- 从头推导离散变分自编码器的数学原理，不跳步。
- 提出可落地的训练方法，代码已开源。
- 适合想理解离散潜空间原理的研究者或实践者。
变分自编码器（VAEs）是使用神经网络进行概率无监督学习的成熟方法。通常，编码器网络定义一个高斯分布的潜空间，从中采样并传递给解码器网络。该模型通过证据下界优化，以重建输入。近年来，离散潜空间日益流行，被认为更适合文本等数据模态。本文提供了一个严谨而实用的离散变分自编码器入门教程，重点介绍潜变量服从分类分布的VAE。假设读者具备基础数学知识，从第一原理逐步推导每一步。随后，我们给出具体的训练方案，并提供示例实现，托管于 https://github.com/alanjeffares/discreteVAE。
原文摘要 · Abstract (English)
Variational Autoencoders (VAEs) are well-established as a principled approach to probabilistic unsupervised learning with neural networks. Typically, an encoder network defines the parameters of a Gaussian distributed latent space from which we can sample and pass realizations to a decoder network. This model is trained to reconstruct its inputs and is optimized through the evidence lower bound. In recent years, discrete latent spaces have grown in popularity, suggesting that they may be a natural choice for many data modalities (e.g. text). In this tutorial, we provide a rigorous, yet practical, introduction to discrete variational autoencoders -- specifically, VAEs in which the latent space is made up of latent variables that follow a categorical distribution. We assume only a basic mathematical background with which we carefully derive each step from first principles. From there, we develop a concrete training recipe and provide an example implementation, hosted at https://github.com/alanjeffares/discreteVAE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。