arXiv:2604.23804cs.LG2026-04

提出拓扑重参数化新方法,让变分自编码器在非平凡流形上也能高效训练。

Reparameterization through Coverings and Topological Weight Priors

论文配图:Reparameterization through Coverings and Topological Weight Priors
图 1 · 摘自论文原文
  • 通过覆盖映射实现非平凡拓扑潜空间的重参数化
  • 在克莱因瓶拓扑潜空间上成功训练出KleinVAE模型
  • 为卷积视觉模型提供拓扑感知的权重先验,适合结构敏感任务

本文将变分自编码器(VAE)中的重参数化技巧推广至具有非平凡拓扑的潜空间——即基流形被其他流形覆盖的情形。由于覆盖映射是可测的,我们建立了基流形上推前密度间的KL散度与覆盖空间上拉回密度间KL散度的不等式关系,在某些情况下使VAE的ELBO中KL项解析可解,即便潜空间拓扑复杂。该方法虽与李群上的重参数化路径相似,但更具一般性;我们将其称为覆盖重参数化(RVC)。通过构造潜空间为克莱因瓶拓扑的KleinVAE模型,成功学习了一个人工数据集。进一步讨论了此类拓扑感知生成模型作为贝叶斯学习中权重先验的应用潜力,尤其适用于卷积视觉模型,因克莱因瓶拓扑曾被证明在视觉表征中有特殊意义。

原文摘要 · Abstract (English)

We generalise the reparameterization trick (RT) applied in variational autoencoders (VAEs) letting these have latent spaces of non-trivial topology - i.e. that of base manifolds covered with other ones, on which some technique for RT is available. That is possible since covering maps are measurable - moreover, this allows to establish an inequality on KL-divergence between pushforward (PF) densities on the base latent manifold, bounding it with KL-divergence between pullbacks on the cover, in some cases making the KL-term of VAE's ELBO analytically tractable, despite the topological non-triviality of the supporting latent manifold. Our development follows a route close but somewhat alternative to reparameterization on Lie groups, the latest proposal for which is to reparameterize PFs of normal densities from the Lie algebra - "through" the exponential map, seen by us as a particular case of what we propose to call reparameterization via covering (RVC). We demonstrate the working of our approach by constructing a VAE with the latent space of Klein bottle (not a Lie group) topology, which we call KleinVAE, successfully learning an appropriate artificial dataset. We discuss potential applicability of such topology-informed generative models as weight priors in Bayesian learning, particularly for convolutional vision models, where said manifold was peculiarly shown to have some relevance.

生成模型拓扑学习变分推断贝叶斯深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。