arXiv:2410.14092cs.LGmath.OC2024-10ICLR被引 5

通过分块对角化加速稀疏PCA,大幅提升计算效率。

Efficient Sparse PCA via Block-Diagonalization

  • 将协方差矩阵重构为分块对角形式,分块求解稀疏PCA。
  • 相比原有方法,平均提速100倍,误差仅0.61%。
  • 适合大规模数据的高效稀疏主成分分析,尤其适用于精确算法改进。

稀疏主成分分析(Sparse PCA)是数据分析与降维的关键工具,但其理论与实践均具挑战性:已被证明为NP难问题,现有精确方法通常需指数级运行时间。本文提出一种新框架,通过(i)将一般输入协方差矩阵近似为重排序的分块对角矩阵,(ii)在每个分块中求解稀疏PCA子问题,(iii)重建原问题解。该框架简洁而强大,可兼容任意现成稀疏PCA算法,实现显著计算加速,附加误差线性依赖于分块对角矩阵的逼近误差。设 $g(k, d)$ 为求解维度 $d$、稀疏常数 $k$ 的稀疏PCA算法的运行时间,则本框架集成后运行时间为 $/mathcal{O}ig( rac{d}{d^ullet} imes g(k, d^ullet) + d^2ig)$,其中 $d^ullet \ eq d$ 为最大块尺寸。例如,结合分支定界算法时,复杂度由 $g(k, d) = /mathcal{O}(k^3�^k)$ 降至 $/mathcal{O}(k^3� �^ullet^{k-1})$,若 $d^ullet$ 较小则实现指数级加速。我们在多个真实世界数据集上进行大规模评估:对于精确稀疏PCA算法,本方法平均提速100.50倍,平均近似误差0.61%;对于近似算法,平均提速6.00倍,平均误差-0.91%,表明方法常能找到更优解。

原文摘要 · Abstract (English)

Sparse Principal Component Analysis (Sparse PCA) is a pivotal tool in data analysis and dimensionality reduction. However, Sparse PCA is a challenging problem in both theory and practice: it is known to be NP-hard and current exact methods generally require exponential runtime. In this paper, we propose a novel framework to efficiently approximate Sparse PCA by (i) approximating the general input covariance matrix with a re-sorted block-diagonal matrix, (ii) solving the Sparse PCA sub-problem in each block, and (iii) reconstructing the solution to the original problem. Our framework is simple and powerful: it can leverage any off-the-shelf Sparse PCA algorithm and achieve significant computational speedups, with a minor additive error that is linear in the approximation error of the block-diagonal matrix. Suppose $g(k, d)$ is the runtime of an algorithm (approximately) solving Sparse PCA in dimension $d$ and with sparsity constant $k$. Our framework, when integrated with this algorithm, reduces the runtime to $\mathcal{O}\left(\frac{d}{d^\star} \cdot g(k, d^\star) + d^2\right)$, where $d^\star \leq d$ is the largest block size of the block-diagonal matrix. For instance, integrating our framework with the Branch-and-Bound algorithm reduces the complexity from $g(k, d) = \mathcal{O}(k^3\cdot d^k)$ to $\mathcal{O}(k^3\cdot d \cdot (d^\star)^{k-1})$, demonstrating exponential speedups if $d^\star$ is small. We perform large-scale evaluations on many real-world datasets: for exact Sparse PCA algorithm, our method achieves an average speedup factor of 100.50, while maintaining an average approximation error of 0.61%; for approximate Sparse PCA algorithm, our method achieves an average speedup factor of 6.00 and an average approximation error of -0.91%, meaning that our method oftentimes finds better solutions.

稀疏PCA降维加速算法分块对角化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。