提出对重尾数据有效的鲁棒PCA方法,突破传统方法对方差的依赖。
Heavy-Tailed Principal Component Analysis
- 用对数损失重构PCA,避免依赖二阶矩
- 在重尾数据下准确恢复主方向,显著优于经典PCA
- 适合含异常值或脉冲噪声的高维数据处理
主成分分析(PCA)是降维的核心工具,但其经典形式依赖二阶矩,在重尾数据和脉冲噪声下表现脆弱。尽管已有多种鲁棒PCA方法,多数仍假设方差有限、依赖稀疏性分解,或通过代理损失函数实现鲁棒性,缺乏对无穷方差模型的统一处理。本文研究基于超统计相关模型 $\mathbf{X} = A^{1/2}\mathbf{G}$ 的高维数据,其中 $A$ 为正随机标量,$\mathbf{G}$ 为高斯向量,该框架涵盖多元t分布及亚高斯α稳定分布等广泛重尾分布。我们采用对数损失进行PCA,即使在矩不存在时仍可定义。理论结果表明,该损失下的主成分与对底层高斯生成器协方差矩阵应用标准PCA所得结果一致。基于此,我们提出了直接从重尾数据估计该协方差矩阵的鲁棒方法,并与经验协方差和Tyler散度估计器对比。大量实验(包括背景去噪任务)表明,所提方法能可靠恢复主方向,在重尾和脉冲噪声下显著优于经典PCA,且在高斯噪声下保持竞争力。
原文摘要 · Abstract (English)
Principal Component Analysis (PCA) is a cornerstone of dimensionality reduction, yet its classical formulation relies critically on second-order moments and is therefore fragile in the presence of heavy-tailed data and impulsive noise. While numerous robust PCA variants have been proposed, most either assume finite variance, rely on sparsity-driven decompositions, or address robustness through surrogate loss functions without a unified treatment of infinite-variance models. In this paper, we study PCA for high-dimensional data generated according to a superstatistical dependent model of the form $\mathbf{X} = A^{1/2}\mathbf{G}$, where $A$ is a positive random scalar and $\mathbf{G}$ is a Gaussian vector. This framework captures a wide class of heavy-tailed distributions, including multivariate $t$ and sub-Gaussian $α$-stable laws. We formulate PCA under a logarithmic loss, which remains well defined even when moments do not exist. Our main theoretical result shows that, under this loss, the principal components of the heavy-tailed observations coincide with those obtained by applying standard PCA to the covariance matrix of the underlying Gaussian generator. Building on this insight, we propose robust estimators for this covariance matrix directly from heavy-tailed data and compare them with the empirical covariance and Tyler's scatter estimator. Extensive experiments, including background denoising tasks, demonstrate that the proposed approach reliably recovers principal directions and significantly outperforms classical PCA in the presence of heavy-tailed and impulsive noise, while remaining competitive under Gaussian noise.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。