PCA虽保留99.9999%方差，却可能抹去关键罕见故障信号，导致决策崩溃。
The Risk Shadow of Principal Component Analysis: When 99.9999% Variance Preservation Causes Catastrophic Decision Errors
- 用期望损失重加权重构协方差，捕捉高影响事件
- 在信用卡欺诈数据上，新方法显著提升罕见事件检测率
- 适用于金融、医疗等高风险场景的鲁棒降维
主成分分析（PCA）旨在保留方差，而非检测罕见灾难性事件所需信息。本文证明了‘风险阴影’的存在：即使保留超过99.9999%的总方差，PCA仍可能完全消除关于罕见高影响故障的所有信号。此时，基于PCA表示的最佳分类器退化为常数预测。根本原因在于方差最大化与尾部风险感知之间的根本错配。为此，我们提出期望值主成分分析（ExPCA）和尾部保持主成分分析（TP-PCA），通过将数据协方差重新加权以关注高影响事件。理论上证明，ExPCA严格优于PCA，能更好保留罕见事件信息，并在合成数据和真实信用卡欺诈检测基准上验证。研究呼吁在高风险决策中重新思考基于方差的降维范式。
原文摘要 · Abstract (English)
Principal Component Analysis (PCA) preserves variance, not the information needed to detect rare catastrophic events. This paper proves the existence of a {\it Risk Shadow}: PCA can retain over 99.9999 percent of total variance while completely erasing all signal about rare, high-impact failures. When this happens, even the best possible classifier operating on the PCA representation reduces to a constant predictor. The root cause is a fundamental mismatch between variance maximization and tail risk awareness. To break the shadow, we introduce Expectile PCA (ExPCA) and Tail-Preserving PCA (TP-PCA), two methods that reweight the data covariance toward high-impact events. We prove theoretically that ExPCA strictly outperforms PCA in retaining rare-event information, and we validate our claims on synthetic data and a real-world credit card fraud detection benchmark. Our results call for a fundamental rethinking of variance-based dimensionality reduction in high-stakes decisions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。