用统计物理视角揭示变分自编码器后验坍缩是相变现象。
Posterior Collapse as a Phase Transition in Variational Autoencoders
- 从稳定性和相变角度分析后验坍缩,提出临界条件
- 当解码器方差超过数据协方差最大特征值时发生坍缩
- 适用于研究生成模型训练机制与潜空间表征能力
我们从统计物理视角研究变分自编码器(VAEs)中的后验坍缩现象,发现其是由数据结构与模型超参数共同决定的相变过程。通过分析坍缩对应平凡解的稳定性,我们确定了一个关键超参数阈值。特别地,推导出坍缩发生的明确判据:当解码器方差超过数据协方差矩阵的最大特征值时,后验坍缩发生。该临界边界表现为近似后验与先验之间KL散度的不连续性,其及其导数呈现明显的非解析行为。我们在合成和真实数据集上验证了这一临界行为,实验结果与理论预测高度一致,表明该坍缩判据在多种VAE架构中具有鲁棒性。基于稳定性的分析表明,后验坍缩并非单纯的优化失败,而是数据结构与变分约束相互作用所引发的涌现相变。这一视角为深度生成模型的可训练性与表征能力提供了新理解。
原文摘要 · Abstract (English)
We investigate the phenomenon of posterior collapse in variational autoencoders (VAEs) from the perspective of statistical physics, and reveal that it constitutes a phase transition governed jointly by data structure and model hyper-parameters. By analyzing the stability of the trivial solution associated with posterior collapse, we identify a critical hyper-parameter threshold. In particular, we derive an explicit criterion for the onset of collapse: posterior collapse occurs when the decoder variance exceeds the largest eigenvalue of the data covariance matrix. This critical boundary, separating meaningful latent inference from collapse, is characterized by a discontinuity in the KL divergence between the approximate posterior and the prior distribution, where the KL divergence and its derivatives exhibit clear non-analytic behavior. We validate this critical behavior on both synthetic and real-world datasets, confirming the existence of a phase transition. The experimental results align well with our theoretical predictions, demonstrating the robustness of our collapse criterion across various VAE architectures. Our stability-based analysis demonstrate that posterior collapse is not merely an optimization failure, but rather an emerging phase transition arising from the interplay between data structure and variational constraints. This perspective offers new insights into the trainability and representational capacity of deep generative models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。