提出可同时处理异常值与缺失数据的高维协方差估计新方法
Cellwise and Casewise Robust Covariance in High Dimensions
- 基于主子空间分解与鲁棒PCA思想，分离并稳定协方差估计
- 在含异常值和缺失数据场景下，性能优于现有方法
- 适合高维数据分析、异常检测等实际应用
样本协方差矩阵是多元统计的核心工具，但对异常值极为敏感，包括案例型异常（来自不同群体）和单元型异常（数据矩阵中个别条目偏离）。尽管已有部分鲁棒协方差估计器能处理两类异常，但计算仅适用于不超过20维。为此，本文提出cellRCov方法，可同时应对案例型异常、单元型异常及缺失数据。该方法基于协方差在主子空间与正交子空间的分解，结合近期鲁棒PCA成果，并采用岭型正则化稳定估计结果。理论分析表明其具有良好的影响函数性质，且满足一致性和渐近正态性。模拟研究证实，在污染和缺失数据场景下，cellRCov表现更优。实际应用展示其在异常检测中的有效性。此外，还构建了用于鲁棒正则化典型相关分析的cellRCCA方法。
原文摘要 · Abstract (English)
The sample covariance matrix is a cornerstone of multivariate statistics, but it is highly sensitive to outliers. These can be casewise outliers, such as cases belonging to a different population, or cellwise outliers, which are deviating cells (entries) of the data matrix. Recently some robust covariance estimators have been developed that can handle both types of outliers, but their computation is only feasible up to at most 20 dimensions. To remedy this we propose the cellRCov method, a robust covariance estimator that simultaneously handles casewise outliers, cellwise outliers, and missing data. It relies on a decomposition of the covariance on principal and orthogonal subspaces, leveraging recent work on robust PCA. It also employs a ridge-type regularization to stabilize the estimated covariance matrix. We establish some theoretical properties of cellRCov, including its casewise and cellwise influence functions as well as consistency and asymptotic normality. A simulation study demonstrates the superior performance of cellRCov in contaminated and missing data scenarios. Furthermore, its practical utility is illustrated in a real-world application to anomaly detection. We also construct and illustrate the cellRCCA method for robust and regularized canonical correlation analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。