用哈密顿系统与跳跃法求解稀疏PCA,提升可解释性与识别准确率
Solve sparse PCA problem by employing Hamiltonian system and leapfrog method
- 通过平滑L1正则化实现稀疏性,用几何积分求解哈密顿系统
- 在人脸数据上分类准确率高于传统PCA,k近邻与核岭回归均表现更优
- 适合需要特征选择与高可解释性的机器学习任务
主成分分析(PCA)广泛用于降维,但其依赖所有特征的稠密组合,缺乏可解释性。本文提出一种新型稀疏PCA算法,采用平滑L1正则化引入稀疏性,并利用哈密顿形式结合几何积分技术求解。具体实现两种数值方法:基于近端梯度(ISTA)的方案,以及基于跳跃法(四阶龙格-库塔)的方案,以最小化兼顾方差最大化与稀疏性约束的能量函数。为提取一组稀疏主成分,进一步引入去噪技术,将原始高维人脸数据映射至低维特征空间。在人脸识别数据集上的实验表明,使用k近邻与核岭回归分类器时,所提方法分类准确率持续优于传统PCA。未来工作将拓展该框架,融合稀疏PCA与现代深度学习架构,应用于多模态识别任务。
原文摘要 · Abstract (English)
Principal Component Analysis (PCA) is a widely utilized technique for dimensionality reduction; however, its inherent lack of interpretability-stemming from dense linear combinations of all feature-limits its applicability in many domains. In this paper, we propose a novel sparse PCA algorithm that imposes sparsity through a smooth L1 penalty and leverages a Hamiltonian formulation solved via geometric integration techniques. Specifically, we implement two distinct numerical methods-one based on the Proximal Gradient (ISTA) approach and another employing a leapfrog (fourth-order Runge-Kutta) scheme-to minimize the energy function that balances variance maximization with sparsity enforcement. To extract a subset of sparse principal components, we further incorporate a deflation technique and subsequently transform the original high-dimensional face data into a lower-dimensional feature space. Experimental evaluations on a face recognition dataset-using both k-nearest neighbor and kernel ridge regression classifiers-demonstrate that the proposed sparse PCA methods consistently achieve higher classification accuracy than conventional PCA. Future research will extend this framework to integrate sparse PCA with modern deep learning architectures for multimodal recognition tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。