从对偶视角重新理解PCA，揭示其与自注意力的联系及优化新方法。
Rethinking PCA Through Duality
- 基于凸差框架，提出可核化且支持外样本的新PCA形式
- 发现经典QR算法本质是差分凸算法，提供优化新视角
- 设计新算法并验证鲁棒PCA在$ l_1 $误差下的优势
受自注意力与（核）主成分分析（PCA）之间新发现关联的启发，本文重新审视了PCA的基本原理。利用差分凸（DC）框架，我们提出了若干新颖的公式，并提供了新的理论见解。特别地，我们证明了一类类似PCA的问题具有可核化性和外样本适用性。此外，我们揭示出同时迭代法——与经典QR算法相关——是差分凸算法（DCA）的一个实例，为这一长期存在的方法提供了优化视角。我们还描述了用于PCA的新算法，并通过实验将其与最先进方法进行了比较。最后，我们引入了一种可核化的对偶形式，用于一种鲁棒型PCA，该方法最小化重构误差的$ l_1 $偏差。
原文摘要 · Abstract (English)
Motivated by the recently shown connection between self-attention and (kernel) principal component analysis (PCA), we revisit the fundamentals of PCA. Using the difference-of-convex (DC) framework, we present several novel formulations and provide new theoretical insights. In particular, we show the kernelizability and out-of-sample applicability for a PCA-like family of problems. Moreover, we uncover that simultaneous iteration, which is connected to the classical QR algorithm, is an instance of the difference-of-convex algorithm (DCA), offering an optimization perspective on this longstanding method. Further, we describe new algorithms for PCA and empirically compare them with state-of-the-art methods. Lastly, we introduce a kernelizable dual formulation for a robust variant of PCA that minimizes the $l_1$ deviation of the reconstruction errors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。