为每个隐变量设计自适应混合高斯先验,提升非线性独立成分分析的解耦效果。
PDGMM-VAE: A Variational Autoencoder with Adaptive Per-Dimension Gaussian Mixture Model Priors for Nonlinear ICA
- 每个隐空间维度用独立的自适应混合高斯先验建模源信号分布
- 在非线性混合任务中有效恢复出源信号并拟合非高斯边缘分布
- 适合需要显式解耦源成分的生成建模与盲源分离场景
独立成分分析是盲源分离中的核心框架,旨在从观测混合信号中恢复满足统计独立性的潜在源信号。本文提出PDGMM-VAE,一种面向源信号的变分自编码器,其中每个隐变量维度(被明确解释为单一源成分)均配置其自身的自适应高斯混合模型先验。该框架引入异质的逐维先验约束,使不同隐空间维度可在统一的概率编码-解码架构下分别建模不同的非高斯源边缘分布。这些源特异性GMM先验参数并非预先设定,而是与编码器、解码器一起通过整体训练目标联合学习。除了模型构建外,我们还提供了理论分析,说明为何这种逐维自适应先验设计在此设置中具有意义:首先,异质先验相比同质共享先验能降低隐变量排列对称性;其次,自适应GMM先验带来的KL正则化会引发源特异性吸引行为,有助于解释训练中各源的专门化现象。我们还阐明了所提模型与标准VAE的关系,并在理想化的线性低噪声条件下给出了弱可恢复性结论。在线性和非线性混合问题上的实验结果表明,PDGMM-VAE能有效恢复潜在源信号并拟合源特异性非高斯边缘分布。结果表明,自适应逐维混合先验设计为基于VAE的ICA和源导向生成建模提供了一个原理清晰且前景可观的方向。
原文摘要 · Abstract (English)
Independent component analysis is a core framework within blind source separation for recovering latent source signals from observed mixtures under statistical independence assumptions. In this work, we propose PDGMM-VAE, a source-oriented variational autoencoder in which each latent dimension, interpreted explicitly as an individual source component, is assigned its own adaptive Gaussian mixture model prior. The proposed framework imposes heterogeneous per-dimension prior constraints, enabling different latent dimensions to model different non-Gaussian source marginals within a unified probabilistic encoder-decoder architecture. The parameters of these source-specific GMM priors are not fixed in advance, but are jointly learned together with the encoder and decoder under the overall training objective. Beyond the model construction itself, we provide a theoretical analysis clarifying why adaptive per-dimension prior design is meaningful in this setting. In particular, we show that heterogeneous per-dimension priors reduce latent permutation symmetry relative to homogeneous shared priors, and we further show that the KL regularization induced by the adaptive GMM prior creates source-specific attraction behavior that helps explain source-wise specialization during training. We also clarify the relation of the proposed model to the standard VAE and provide a weak recovery statement in an idealized linear low-noise regime. Experimental results on both linear and nonlinear mixing problems show that PDGMM-VAE can recover latent source signals and fit source-specific non-Gaussian marginals effectively. These results suggest that adaptive per-dimension mixture-prior design provides a principled and promising direction for VAE-based ICA and source-oriented generative modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。