解释了β-VAE解耦失败的根本原因,并提出新模型稳定解耦性能。
Mutual Information Collapse Explains Disentanglement Failure in $β$-VAEs
- 发现强正则化导致潜在变量互信息坍缩,破坏语义信息。
- 在β>1时,编码器增益谱收缩,使潜在因子互信息归零。
- 引入λβ-VAE,用额外重构项避免信息坍缩,提升解耦稳定性。
β-VAE是无监督解耦学习的基础框架,通过调节β平衡潜在变量的独立性与重建精度。然而,实验中解耦性能呈现普遍的非单调趋势:如MIG和SAP指标在中等β值达到峰值后随β增大而下降。本文证明该现象是根本性的信息论失效——过强的KL正则化压力促使潜在变量边缘独立,牺牲了其语义信息量。在线性高斯设定下,我们证明当β>1时,稳态动力学引发编码器增益的谱收缩,导致潜在因子间互信息归零。为解决此问题,我们提出λβ-VAE,通过辅助的L2重构惩罚λ将正则化压力与信息坍缩解耦。在dSprites、Shapes3D和MPI3D-real上的大量实验表明,λ>0能显著扩展β的有效范围,稳定解耦性能并恢复潜在表示的信息丰富度,为变分推断骨干中的双参数正则化提供了理论依据。
原文摘要 · Abstract (English)
The $β$-VAE is a foundational framework for unsupervised disentanglement, using $β$ to regulate the trade-off between latent factorization and reconstruction fidelity. Empirically, however, disentanglement performance exhibits a pervasive non-monotonic trend: benchmarks such as MIG and SAP typically peak at intermediate $β$ and collapse as regularization increases. We demonstrate that this collapse is a fundamental information-theoretic failure, where strong Kullback-Leibler pressure promotes marginal independence at the expense of the latent channel's semantic informativeness. By formalizing this mechanism in a linear-Gaussian setting, we prove that for $β> 1$, stationarity-induced dynamics trigger a spectral contraction of the encoder gain, driving latent-factor mutual information to zero. To resolve this, we introduce the $λβ$-VAE, which decouples regularization pressure from informational collapse via an auxiliary $L_2$ reconstruction penalty $λ$. Extensive experiments on dSprites, Shapes3D, and MPI3D-real confirm that $λ> 0$ stabilizes disentanglement and restores latent informativeness over a significantly broader range of $β$, providing a principled theoretical justification for dual-parameter regularization in variational inference backbones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。