分析过定模型下EM算法收敛性，揭示权重初始值对迭代效率的影响。
Characterizing Evolution in Expectation-Maximization Estimates for Overspecified Mixed Linear Regression
- 在过定两分量线性混合模型中，研究EM算法收敛速度与初始权重平衡性的关系。
- 不平衡初始权重时线性收敛（需O(log(1/ε))步），平衡时为亚线性收敛（需O(ε⁻²)步）。
- 结果适用于低信噪比场景，适合关注统计学习理论的研究者。
混合模型因实际效果显著和理论基础完善而备受关注。持续挑战在于模型误设问题，即拟合模型的分量数多于数据分布的真实分量数。本文针对过定两分量混合线性回归（2MLR）在未知d维回归参数和混合权重下的目标误设情形，建立对期望最大化（EM）算法行为的理论理解。在总体层面（定理5.1），若初始混合权重不平衡，则回归参数以线性速度收敛至ε精度，所需步数为O(log(1/ε))；若初始权重平衡，则收敛速度为亚线性，需O(ε⁻²)步达到ε精度。在有限样本层面（定理6.1），当混合权重充分不平衡时，统计精度为O((d/n)¹ᐟ²)；当充分平衡时，精度为O((d/n)¹ᐟ⁴)，其中n为样本数。进一步地，通过将定理5.1中的精度ε设置为匹配定理6.1的有限样本精度，可推导出有限样本层面的迭代复杂度边界：对于不平衡初始权重为O(log(n/d))，平衡时为O((n/d)¹ᐟ²)。最后，还将分析扩展至低信噪比（SNR）情形。
原文摘要 · Abstract (English)
Mixture models have attracted significant attention due to practical effectiveness and comprehensive theoretical foundations. A persisting challenge is model misspecification, which occurs when the model to be fitted has more mixture components than those in the data distribution. In this paper, we develop a theoretical understanding of the Expectation-Maximization (EM) algorithm's behavior in the context of targeted model misspecification for overspecified two-component Mixed Linear Regression (2MLR) with unknown $d$-dimensional regression parameters and mixing weights. In Theorem 5.1 at the population level, with an unbalanced initial guess for mixing weights, we establish linear convergence of regression parameters in $O(\log(1/ε))$ steps. Conversely, with a balanced initial guess for mixing weights, we observe sublinear convergence in $O(ε^{-2})$ steps to achieve the $ε$-accuracy at Euclidean distance. In Theorem 6.1 at the finite-sample level, for mixtures with sufficiently unbalanced fixed mixing weights, we demonstrate a statistical accuracy of $O((d/n)^{1/2})$, whereas for those with sufficiently balanced fixed mixing weights, the accuracy is $O((d/n)^{1/4})$ given $n$ data samples. Furthermore, we underscore the connection between our population level and finite-sample level results: by setting the desired final accuracy $ε$ in Theorem 5.1 to match that in Theorem 6.1 at the finite-sample level, namely letting $ε= O((d/n)^{1/2})$ for sufficiently unbalanced fixed mixing weights and $ε= O((d/n)^{1/4})$ for sufficiently balanced fixed mixing weights, we intuitively derive iteration complexity bounds $O(\log (1/ε))=O(\log (n/d))$ and $O(ε^{-2})=O((n/d)^{1/2})$ at the finite-sample level for sufficiently unbalanced and balanced initial mixing weights. We further extend our analysis in overspecified setting to low SNR regime.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。