自动识别VAE中真正有用的隐变量维度,避免手动试错。
ARD-VAE: A Statistical Formulation to Find the Relevant Latent Dimensions of Variational Autoencoders
- 用分层先验动态估计隐空间各维度方差,找出关键因子。
- 在多个数据集上提升FID分数与解耦性,验证方法有效性。
- 适合想优化模型隐空间结构的研究者或工程师。
变分自编码器(VAE)是一种流行的深度隐变量模型,因其对数据分布建模的简洁有效而广受青睐,且其优化目标比其他深度隐变量模型更易处理。然而,VAE的隐维数常作为超参数凭经验设定,影响模型性能,如发现数据的潜在解释因子。为此,本文提出一种统计方法,用于自动识别建模数据所需的隐变量维度。通过在隐空间引入分层先验,利用编码数据估计各隐轴方差,从而识别出相关维度。具体做法是将原始VAE目标函数中的固定先验替换为该分层先验,其余部分保持不变。所提方法称为自动相关性检测变分自编码器(ARD-VAE)。我们在多个基准数据集上验证了其在发现相关隐维度及提升评估指标(如FID分数、解耦分析)方面的有效性。
原文摘要 · Abstract (English)
The variational autoencoder (VAE) is a popular, deep, latent-variable model (DLVM) due to its simple yet effective formulation for modeling the data distribution. Moreover, optimizing the VAE objective function is more manageable than other DLVMs. The bottleneck dimension of the VAE is a crucial design choice, and it has strong ramifications for the model's performance, such as finding the hidden explanatory factors of a dataset using the representations learned by the VAE. However, the size of the latent dimension of the VAE is often treated as a hyperparameter estimated empirically through trial and error. To this end, we propose a statistical formulation to discover the relevant latent factors required for modeling a dataset. In this work, we use a hierarchical prior in the latent space that estimates the variance of the latent axes using the encoded data, which identifies the relevant latent dimensions. For this, we replace the fixed prior in the VAE objective function with a hierarchical prior, keeping the remainder of the formulation unchanged. We call the proposed method the automatic relevancy detection in the variational autoencoder (ARD-VAE). We demonstrate the efficacy of the ARD-VAE on multiple benchmark datasets in finding the relevant latent dimensions and their effect on different evaluation metrics, such as FID score and disentanglement analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。