揭示混合线性回归中EM算法的轨迹与收敛性,突破传统理论局限。
Structural Properties, Cycloid Trajectories and Non-Asymptotic Guarantees of EM Algorithm for Mixed Linear Regression
- 基于轨迹分析,推导出全未知参数下EM的显式更新表达式。
- 噪声为零时,参数轨迹呈旋轮线;高信噪比下其偏离程度可量化。
- 首次实现任意初值下的有限样本收敛保证,适合优化与统计学习研究者。
本文研究了两组件混合线性回归(2MLR)在未知混合权重和回归参数条件下,期望最大化(EM)算法的结构特性、旋轮线轨迹及非渐近收敛保证。已有研究证明了已知均衡权重下的全局收敛性,以及无噪声和高信噪比(SNR)下的超线性收敛。然而,在完全未知设置下,EM的轨迹行为与收敛阶数仍不明确。本文推导出所有SNR条件下2MLR的显式EM更新表达式,分析其结构特性与旋轮线轨迹。在无噪声情况下,通过建立次优角度的递推关系,证明回归参数的迭代轨迹形成旋轮线;在高SNR下,量化其与旋轮线轨迹的偏差。基于轨迹分析揭示收敛阶:当估计值接近正交于真实值时为线性收敛,当估计与真实值夹角在总体层面较小时为二次收敛。通过精化有限样本与总体EM更新间的统计误差界,将统计精度与次优角度关联,证明在有限样本水平上任意初始化均可实现收敛。本工作为2MLR中EM算法的分析提供了新颖的轨迹驱动框架。
原文摘要 · Abstract (English)
This work investigates the structural properties, cycloid trajectories, and non-asymptotic convergence guarantees of the Expectation-Maximization (EM) algorithm for two-component Mixed Linear Regression (2MLR) with unknown mixing weights and regression parameters. Recent studies have established global convergence for 2MLR with known balanced weights and super-linear convergence in noiseless and high signal-to-noise ratio (SNR) regimes. However, the theoretical behavior of EM in the fully unknown setting remains unclear, with its trajectory and convergence order not yet fully characterized. We derive explicit EM update expressions for 2MLR with unknown mixing weights and regression parameters across all SNR regimes and analyze their structural properties and cycloid trajectories. In the noiseless case, we prove that the trajectory of the regression parameters in EM iterations traces a cycloid by establishing a recurrence relation for the sub-optimality angle, while in high SNR regimes we quantify its discrepancy from the cycloid trajectory. The trajectory-based analysis reveals the order of convergence: linear when the EM estimate is nearly orthogonal to the ground truth, and quadratic when the angle between the estimate and ground truth is small at the population level. Our analysis establishes non-asymptotic guarantees by sharpening bounds on statistical errors between finite-sample and population EM updates, relating EM's statistical accuracy to the sub-optimality angle, and proving convergence with arbitrary initialization at the finite-sample level. This work provides a novel trajectory-based framework for analyzing EM in Mixed Linear Regression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。