arXiv:2510.11953cs.LGcs.AI2025-10

用MMD重构潜在空间,实现更优解耦表征。

Sculpting Latent Spaces With MMD: Disentanglement With Programmable Priors

  • 以MMD替代KL散度,可精准控制潜在空间分布
  • 在CIFAR-10和Tiny ImageNet上达成顶尖解耦效果
  • 支持自定义先验,适合需要语义对齐的模型设计

学习解耦表示是机器学习的核心目标之一,即不同变化因素由独立的潜在变量捕捉。主流方法为变分自编码器(VAE),其通过Kullback-Leibler(KL)散度正则化使潜在空间逼近因子化高斯先验。然而,本文提供直接证据表明,该KL正则化机制不可靠,无法有效约束聚合后验分布。我们提出新型无监督潜在可预测性评分(LPS)验证并量化此问题导致的纠缠程度。为此,我们引入可编程先验框架,基于最大均值差异(MMD)构建,使从业者能显式塑造潜在空间,在不牺牲重建质量的前提下,于复杂数据集如CIFAR-10和Tiny ImageNet上实现当前最优的互独立性。此外,我们展示如何利用该可编程性设计复杂先验,提升与语义特征的对齐。本工作为表示工程提供了基础工具,拓展了模型可识别性和因果推理的新路径。

原文摘要 · Abstract (English)

Learning disentangled representations, where distinct factors of variation are captured by independent latent variables, is a central goal in machine learning. The dominant approach has been the Variational Autoencoder (VAE) framework, which uses a Kullback-Leibler (KL) divergence penalty to encourage the latent space to match a factorized Gaussian prior. In this work, however, we provide direct evidence that this KL-based regularizer is an unreliable mechanism, consistently failing to enforce the target distribution on the aggregate posterior. We validate this and quantify the resulting entanglement using our novel, unsupervised Latent Predictability Score (LPS). To address this failure, we introduce the Programmable Prior Framework, a method built on the Maximum Mean Discrepancy (MMD). Our framework allows practitioners to explicitly sculpt the latent space, achieving state-of-the-art mutual independence on complex datasets like CIFAR-10 and Tiny ImageNet without the common reconstruction trade-off. Furthermore, we demonstrate how this programmability can be used to engineer sophisticated priors that improve alignment with semantically meaningful features. Ultimately, our work provides a foundational tool for representation engineering, opening new avenues for model identifiability and causal reasoning.

解耦表征潜在空间MMDVAE改进

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。