arXiv:2505.24684cs.LGcs.AI2025-05

发现因子化VAE中隐含的解耦粒度机制,可影响特征解耦效果。

Disentangling Granularity: An Implicit Inductive Bias in Factorized VAEs

  • 提出解耦粒度作为因子化VAE的隐式先验偏差
  • 实验验证解耦粒度影响高低复杂度特征的解耦能力
  • 适合研究生成模型可解释性与潜在变量结构的研究者

尽管变分自编码器(VAEs)及其变体在无监督学习语义有意义的解耦表示方面取得成功,但其面临根本性理论挑战:缺乏隐式先验偏差时,无监督解耦难以实现。本文聚焦于具有因子化先验的VAE中的隐式先验偏差。通过分析η-TCVAE中的总相关性,我们揭示了一种关键的隐式先验偏差——解耦粒度,并发现参数空间中证据下界(ELBO)存在有趣的“V”形最优轨迹。该发现通过超过10万次实验及新提出的η-STCVAE模型得到验证。结果表明,传统因子化VAE受限于固定解耦粒度,倾向于解耦低复杂度特征;而通过η-STCVAE适当调节解耦粒度,可扩展解耦范围,实现高复杂度特征的解耦。本研究揭示了解耦粒度作为因子化VAE的隐式先验偏差,同时影响解耦性能与ELBO推断,为理解VAE的可解释性与内在偏差提供了新视角。

原文摘要 · Abstract (English)

Despite the success in learning semantically meaningful, unsupervised disentangled representations, variational autoencoders (VAEs) and their variants face a fundamental theoretical challenge: substantial evidence indicates that unsupervised disentanglement is unattainable without implicit inductive bias, yet such bias remains elusive. In this work, we focus on exploring the implicit inductive bias that drive disentanglement in VAEs with factorization priors. By analyzing the total correlation in \b{eta}-TCVAE, we uncover a crucial implicit inductive bias called disentangling granularity, which leads to the discovery of an interesting "V"-shaped optimal Evidence Lower Bound (ELBO) trajectory within the parameter space. This finding is validated through over 100K experiments using factorized VAEs and our newly proposed model, \b{eta}-STCVAE. Notably, experimental results reveal that conventional factorized VAEs, constrained by fixed disentangling granularity, inherently tend to disentangle low-complexity feature. Whereas, appropriately tuning disentangling granularity, as enabled by \b{eta}-STCVAE, broadens the range of disentangled representations, allowing for the disentanglement of high-complexity features. Our findings unveil that disentangling granularity as an implicit inductive bias in factorized VAEs influence both disentanglement performance and the inference of the ELBO, offering fresh insights into the interpretability and inherent biases of VAEs.

VAE解耦表征隐式先验可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。