提出新算法实现两类混合线性回归的全局收敛与最优聚类。
Learning a Class of Mixed Linear Regressions: Global Convergence under General Data Conditions
- 分两步递归估计参数方向与尺度,结合最小二乘与EM原理。
- 在弱于i.i.d.和持久激励的条件下实现全局收敛,收敛速度可证。
- 无需激励条件即可实现渐近最优数据聚类,适合复杂数据场景。
混合线性回归(MLR)因其能通过多个线性子模型捕捉非线性关系而受到广泛关注。尽管已有大量研究致力于该系统的参数估计与标签识别,但多数方法采用离线算法,依赖严格的独立同分布(i.i.d.)或持久激励(PE)条件,且仅能保证局部收敛。本文研究一类双成分随机混合线性回归的递归估计与数据聚类问题。针对其固有的非凸优化难题,提出一种新颖的两步递归识别算法:利用最小二乘法估计未知参数的方向向量,通过期望最大化(EM)原则估计缩放系数。在弱于传统i.i.d.和PE条件的一般数据假设下,首次证明了所提算法的全局收敛性及收敛速率。进一步证明,在无需任何激励条件的前提下,数据聚类性能——包括累积误分类误差和组内误差——可渐近达到最优。最后,通过数值实验验证了算法性能。
原文摘要 · Abstract (English)
Mixed linear regression (MLR) has attracted increasing attention because of its great theoretical and practical importance in capturing nonlinear relationships by utilizing a mixture of linear regression sub-models. Although considerable efforts have been devoted to the learning problem of such systems, i.e., estimating data labels and identifying model parameters, most existing investigations employ the offline algorithm, impose the strict independent and identically distributed (i.i.d.) or persistent excitation (PE) conditions on the regressor data, and provide local convergence results only. In this paper, we investigate the recursive estimation and data clustering problems for a class of stochastic MLRs with two components. To address this inherently nonconvex optimization problem, we propose a novel two-step recursive identification algorithm to estimate the true parameters, where the direction vector and the scaling coefficient of the unknown parameters are estimated by the least squares and the expectation-maximization (EM) principles, respectively. Under a general data condition, which is much weaker than the traditional i.i.d. and PE conditions, we establish the global convergence and the convergence rate of the proposed identification algorithm for the first time. Furthermore, we prove that, without any excitation condition on the regressor data, the data clustering performance including the cumulative mis-classification error and the within-cluster error can be optimal asymptotically. Finally, we provide a numerical example to illustrate the performance of the proposed learning algorithm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。