改进噪声幂法分析,实现去中心化PCA的加速收敛。
Improved Analysis of the Accelerated Noisy Power Method with Applications to Decentralized PCA
- 提出更宽松的噪声条件下的加速噪声幂法新分析
- 证明该方法收敛率不可再提升,且噪声条件无法放宽
- 首个具备理论保证的加速去中心化PCA算法
我们分析了加速噪声幂法,一种在仅能获得近似矩阵-向量乘积场景下进行主成分分析的算法,此类情形常见于去中心化PCA。此前研究虽表明加速可提升收敛速度,但其理论保证依赖过强的扰动上界,限制了实际应用。本文提供新的分析,使加速收敛在更宽松的噪声条件下仍成立。我们证明该分析在最坏情况下是优化的,即收敛率无法进一步提升,且所推导的噪声条件无法放松而不丧失收敛性。通过该结果,我们设计出一种通信成本与非加速方法相当的加速去中心化PCA算法。据我们所知,这是首个具有理论保证的加速去中心化PCA算法。
原文摘要 · Abstract (English)
We analyze the Accelerated Noisy Power Method, an algorithm for Principal Component Analysis in the setting where only inexact matrix-vector products are available, which can arise for instance in decentralized PCA. While previous works have established that acceleration can improve convergence rates compared to the standard Noisy Power Method, these guarantees require overly restrictive upper bounds on the magnitude of the perturbations, limiting their practical applicability. We provide an improved analysis of this algorithm, which preserves the accelerated convergence rate under much milder conditions on the perturbations. We show that our new analysis is worst-case optimal, in the sense that the convergence rate cannot be improved, and that the noise conditions we derive cannot be relaxed without sacrificing convergence guarantees. We demonstrate the practical relevance of our results by deriving an accelerated algorithm for decentralized PCA, which has similar communication costs to non-accelerated methods. To our knowledge, this is the first decentralized algorithm for PCA with provably accelerated convergence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。