用低秩混合模型优化生成流的潜在分布，提升训练速度与生成质量。
Simplifying Flow Matching Transformations with Low-Rank Mixture Models
- 用MPPCA替代标准正态分布做潜在空间分布，减少数据与潜空间的拓扑差异。
- 在表格和图像数据上，训练收敛更快，生成质量优于基线模型。
- MPPCA可快速用EM算法拟合，适合高维生成任务中的初始化。
归一化流是强大的生成模型，通过学习复杂数据分布与简单潜分布之间的可逆映射来建模数据。然而，传统选择标准正态作为潜分布可能导致学习到的流变换过于复杂，因数据与潜空间分布存在拓扑不匹配，导致训练变慢且性能下降。本文提出使用概率主成分分析混合模型（MPPCA）作为归一化流的潜分布。通过学习更贴近数据分布的潜分布（以KL散度衡量），简化了流变换，从而实现更快的收敛和更好的生成性能。关键优势在于，MPPCA可通过期望最大化算法快速、低成本地拟合，使其在高维生成任务中具备实用性。我们在表格和图像数据集上验证了该方法，结果表明相比基线模型，在训练效率和生成质量上均有持续提升。
原文摘要 · Abstract (English)
Normalizing flows are powerful generative models that learn an invertible mapping between complex data distributions and simple latent distributions, typically a standard normal density. However, this choice of latent density can impose unnecessary complexity on the learned flow transformation due to the topological mismatch between the latent and data densities, leading to slower training and suboptimal performance. In this work, we propose using mixtures of probabilistic principal component analyzers (MPPCA) as the latent density for normalizing flows. We simplify the learned flow transformation by learning a latent distribution that more closely aligns with the data distribution in terms of KL divergence, thus enabling faster convergence and improved generative performance. Critically, MPPCA models can be fit quickly and cheaply using the expectation-maximization algorithm, making them a practical choice for initializing latent distributions even in high-dimensional generative tasks. We validate our method on both tabular and image datasets, demonstrating consistent gains in training efficiency and generation quality compared to baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。