提出一种新型高斯混合模型，用分段常数特征值结构提升灵活性与简洁性。
Parsimonious Gaussian mixture models with piecewise-constant eigenvalue profiles
- 采用分段常数特征值的协方差结构，平衡复杂度与表达能力。
- 在密度拟合、聚类和去噪任务中均实现更高似然与更少参数的权衡。
- 适合需要高效建模高维非各向同性分布的研究者或工程应用。
高斯混合模型（GMM）广泛应用于统计学习，尤其在无监督问题中。全参数化GMM在高维空间中存在协方差矩阵过参数化问题，而球形GMM（各向同性协方差）则缺乏拟合各向异性分布的能力。本文提出一类新的简约型GMM，其协方差矩阵具有分段常数特征值结构，可扩展经典的概率主成分分析混合模型（MPPCA）等低秩模型，支持任意特征值重数序列。若特征值重数预先设定，可自然推导出期望最大化（EM）算法以学习混合参数；否则，针对联合学习混合参数与超参数这一难题，提出逐分量正则化EM算法，并证明其单调性。在多种无监督实验中，包括密度拟合、聚类和单图像去噪，该模型展现出更优的似然-简洁性权衡。
原文摘要 · Abstract (English)
Gaussian mixture models (GMMs) are ubiquitous in statistical learning, particularly for unsupervised problems. While full GMMs suffer from the overparameterization of their covariance matrices in high-dimensional spaces, spherical GMMs (with isotropic covariance matrices) certainly lack flexibility to fit certain anisotropic distributions. Connecting these two extremes, we introduce a new family of parsimonious GMMs with piecewise-constant covariance eigenvalue profiles. These extend several low-rank models like the celebrated mixtures of probabilistic principal component analyzers (MPPCA), by enabling any possible sequence of eigenvalue multiplicities. If the latter are prespecified, then we can naturally derive an expectation-maximization (EM) algorithm to learn the mixture parameters. Otherwise, to address the notoriously-challenging issue of jointly learning the mixture parameters and hyperparameters, we propose a componentwise penalized EM algorithm, whose monotonicity is proven. We show the superior likelihood-parsimony tradeoffs achieved by our models on a variety of unsupervised experiments: density fitting, clustering and single-image denoising.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。