arXiv:2412.21203cs.DScs.LG2024-12被引 5

提出高效算法验证稀疏奇异值上界,推动鲁棒统计与子空间分析进展

SoS Certificates for Sparse Singular Values and Their Applications: Robust Statistics, Subspace Distortion, and More

  • 基于平方和(SoS)层次结构设计多项式时间验证算法
  • 在几乎最广范围内实现优于传统奇异值的稀疏上界认证
  • 适用于鲁棒统计、子空间畸变等任务,性能逼近理论下限

我们研究随机矩阵阵列的稀疏奇异值证书。当 $M$ 为 $n \times d$ 独立高斯矩阵时,提出一类新多项式时间算法,可对满足 $\|u\|=1$ 且最多有 $ηn$ 个非零项的单位向量 $u$,认证 $\|M u\|$ 的上界。该基本算法原语贯穿于算法统计学与理论计算机科学诸多问题。所提算法在几乎最广泛范围的 $n, d, η$ 下,给出的上界渐近小于由最大奇异值给出的平凡上界。若能以多项式因子扩展此范围,则违反统计查询(SQ)与低度多项式模型中的下界。算法核心依赖平方和(SoS)层次结构。为证明其正确性,建立图矩阵方法与Efron-Stein分解之间的新组合关联。作为应用,获得一系列新高效算法:在鲁棒统计中,实现均值与协方差估计的新算法,其破绽点与样本复杂度的权衡接近我们建立的SQ与低度多项式下界;同时获得关于 $\mathbb{R}^n$ 中随机子空间的 $\ell_1/\ell_2$ 畸变的多项式时间认证保证,以及稀疏主成分分析与随机矩阵 $2\rightarrow p$ 范数的认证新结果。

原文摘要 · Abstract (English)

We study $\textit{sparse singular value certificates}$ for random rectangular matrices. If $M$ is an $n \times d$ matrix with independent Gaussian entries, we give a new family of polynomial-time algorithms which can certify upper bounds on the maximum of $\|M u\|$, where $u$ is a unit vector with at most $ηn$ nonzero entries for a given $η\in (0,1)$. This basic algorithmic primitive lies at the heart of a wide range of problems across algorithmic statistics and theoretical computer science. Our algorithms certify a bound which is asymptotically smaller than the naive one, given by the maximum singular value of $M$, for nearly the widest-possible range of $n,d,$ and $η$. Efficiently certifying such a bound for a range of $n,d$ and $η$ which is larger by any polynomial factor than what is achieved by our algorithm would violate lower bounds in the SQ and low-degree polynomials models. Our certification algorithm makes essential use of the Sum-of-Squares hierarchy. To prove the correctness of our algorithm, we develop a new combinatorial connection between the graph matrix approach to analyze random matrices with dependent entries, and the Efron-Stein decomposition of functions of independent random variables. As applications of our certification algorithm, we obtain new efficient algorithms for a wide range of well-studied algorithmic tasks. In algorithmic robust statistics, we obtain new algorithms for robust mean and covariance estimation with tradeoffs between breakdown point and sample complexity, which are nearly matched by SQ and low-degree polynomial lower bounds (that we establish). We also obtain new polynomial-time guarantees for certification of $\ell_1/\ell_2$ distortion of random subspaces of $\mathbb{R}^n$ (also with nearly matching lower bounds), sparse principal component analysis, and certification of the $2\rightarrow p$ norm of a random matrix.

稀疏奇异值鲁棒统计子空间分析算法验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。