arXiv:2607.23751cs.LGstat.ML2026-07

通过软约束优化变分自编码器潜空间,平衡表达能力与解耦性。

Soft-Constrained Optimization of Latent Space in Variational Autoencoders

论文配图:Soft-Constrained Optimization of Latent Space in Variational Autoencoders
图 1 · 摘自论文原文
  • 引入熵约束提升单个潜变量的表达能力
  • 在dSprites上使激活得分提升43%-62%,重建误差降低38%
  • 可自动剪枝低熵维度,适合需要紧凑潜空间的任务

变分自编码器(VAE)的性能依赖于潜空间的两个特性:单个潜变量的高表达能力,以及潜变量整体的低维、解耦结构。强化KL正则化会削弱表达能力,而弱化则导致解耦性下降。本文将VAE训练建模为软约束优化问题:首先施加基于熵的约束(EC),证明潜码熵上限决定了其携带生成因子信息的能力;其次提出权重过滤方法,在软约束松弛下剪除低熵维度。在dSprites数据集上,EC使总体潜变量激活得分提升43-62%,在所有η-VAE变体中达到最高FactorVAE分数(0.891 vs 0.847),重建误差降低最多达38%。在MNIST上,权重过滤将潜空间从10维压缩至2维,分类准确率仍超90%,收敛速度比无EC时快37%。此外发现,低熵离散因子倾向于合并至单一潜变量,高熵连续因子则分布于多个潜变量中。

原文摘要 · Abstract (English)

The usefulness of a variational autoencoder (VAE) depends on two properties of its latent space that are hard to obtain together: high encoding capacity in the individual latent variables, and a low-dimensional, disentangled organization of those variables. Weakening the Kullback-Leibler regularization raises capacity but degrades disentanglement, while strengthening it prunes latent variables away entirely. We formulate VAE training as a soft-constrained optimization problem that addresses both. First, we impose an entropy-based constraint (EC) on individual latent variables, showing that the entropy of a latent code upper-bounds the mutual information it carries about the generative factors of the data. Second, we propose a weight-filter method that exploits the slack of the soft constraint to prune low-entropy dimensions during downstream training. On dSprites, the EC raises the aggregate latent-variable activation score by 43-62% over a vanilla VAE, attains the highest FactorVAE score among the \b{eta} \b{eta}-VAE variants (0.891 vs 0.847), and lowers reconstruction error by up to 38%. On MNIST, the weight filter reduces the latent dimensionality supplied to a downstream classifier from ten to two while holding accuracy above 90%, converging in 37% fewer epochs than the same procedure without the EC. We also find that low-entropy discrete factors tend to merge into a single latent variable, whereas high-entropy continuous factors are distributed across several.

变分自编码器潜空间优化解耦表示软约束

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。