EM算法在过参数高斯混合模型中可指数级加速收敛。
Learning Overspecified Gaussian Mixtures Exponentially Fast with the EM Algorithm
- 基于正则单纯形均值结构与非退化权重，分析了群体EM的收敛性。
- 理论证明在O(log(1/ε))次迭代内可达ε精度，远快于传统亚线性速率。
- 适用于高维聚类与密度估计，对初始化和模型设计具指导意义。
本文研究当拟合模型的成分数超过真实分布时，EM算法在过参数高斯混合模型中的收敛性质。聚焦于成分均值位于正则单纯形顶点且混合权重满足非退化条件的结构性配置，我们证明群体EM算法在KL距离下呈指数级收敛。分析利用负对数似然函数在最优解邻域内的强凸性，并结合Polyak-Łojasiewicz不等式，表明ε精度近似可在O(log(1/ε))次迭代内达成。进一步通过推导有限样本下的显式统计收敛保证，将结果扩展至实际场景。合成数据上的数值实验验证了理论结果，凸显收敛速度相较于传统亚线性率的显著提升。本工作不仅深化了对EM在过参数设定下行为的理解，也为高维聚类与密度估计任务中的初始化策略和模型设计提供了实用洞见。
原文摘要 · Abstract (English)
We investigate the convergence properties of the EM algorithm when applied to overspecified Gaussian mixture models -- that is, when the number of components in the fitted model exceeds that of the true underlying distribution. Focusing on a structured configuration where the component means are positioned at the vertices of a regular simplex and the mixture weights satisfy a non-degeneracy condition, we demonstrate that the population EM algorithm converges exponentially fast in terms of the Kullback-Leibler (KL) distance. Our analysis leverages the strong convexity of the negative log-likelihood function in a neighborhood around the optimum and utilizes the Polyak-Łojasiewicz inequality to establish that an $ε$-accurate approximation is achievable in $O(\log(1/ε))$ iterations. Furthermore, we extend these results to a finite-sample setting by deriving explicit statistical convergence guarantees. Numerical experiments on synthetic datasets corroborate our theoretical findings, highlighting the dramatic acceleration in convergence compared to conventional sublinear rates. This work not only deepens the understanding of EM's behavior in overspecified settings but also offers practical insights into initialization strategies and model design for high-dimensional clustering and density estimation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。