详解VAE中高斯分布KL散度的推导过程，助你理解隐空间正则化机制。
KL Divergence Between Gaussians: A Step-by-Step Derivation for the Variational Autoencoder Objective
- 从连续变量定义出发，逐步推导一维与对角协方差多元高斯的闭式解
- 揭示各分量在训练中的正则化作用，解释隐空间结构形成原理
- 适合想深入理解VAE优化目标背后的数学逻辑的研究者
KL散度是信息论中衡量两个概率分布差异的核心概念。在变分自编码器（VAE）中，它作为关键正则项，约束隐空间结构，从而赋予模型生成能力。本文详细推导了高斯分布间KL散度的闭式表达式，该情况在实际VAE实现中尤为常见。从连续随机变量的一般定义出发，首先推导一维情形，再在对角协方差假设下扩展至多维情形。最后讨论结果中各项的含义及其对模型训练动态的影响。
原文摘要 · Abstract (English)
Kullback-Leibler (KL) divergence is a fundamental concept in information theory that quantifies the discrepancy between two probability distributions. In the context of Variational Autoencoders (VAEs), it serves as a central regularization term, imposing structure on the latent space and thereby enabling the model to exhibit generative capabilities. In this work, we present a detailed derivation of the closed-form expression for the KL divergence between Gaussian distributions, a case of particular importance in practical VAE implementations. Starting from the general definition for continuous random variables, we derive the expression for the univariate case and extend it to the multivariate setting under the assumption of diagonal covariance. Finally, we discuss the interpretation of each term in the resulting expression and its impact on the training dynamics of the model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。