arXiv:2411.12438cs.DScs.LG2024-11被引 1

用平方和方法实现非球形高斯混合聚类的降维,提升效率并突破理论下限。

Dimension Reduction via Sum-of-Squares and Improved Clustering Algorithms for Non-Spherical Mixtures

  • 基于平方和方法构造保分离性的低维投影,替代传统SVD降维。
  • 在样本数为poly(n)或d^O(log w_min^{-1})时,可高效聚类任意非球形混合模型。
  • 首次突破统计查询与平方和下限,适用于含异常值的通用高斯混合场景。

我们提出一种新方法,通过基于平方和的子程序对非球形(即任意协方差)高斯混合模型进行聚类。该方法找到输入数据的低维分离保全投影,成为经典球形聚类算法中SVD降维的非球形对应。作为应用,我们得到两个算法:(1) 在n ≥ poly(d) f(w_min^{-1})个样本、poly(n)时间内聚类中心高斯混合;(2) 在n ≥ d^{O(log w_min^{-1})} f(w_min^{-1})个样本、n^{O(log w_min^{-1})}时间内聚类同协方差但未知的k个高斯混合。其中,w_min为最小混合权重,f不依赖维度d。算法天然支持容忍独立于维度的任意异常值。此前最优方法需d^{O(k)} f(w_min^{-1})样本与时间,而本工作首次在广泛情形下突破了已知的d^{Ω(k)}统计查询与平方和下限。

原文摘要 · Abstract (English)

We develop a new approach for clustering non-spherical (i.e., arbitrary component covariances) Gaussian mixture models via a subroutine, based on the sum-of-squares method, that finds a low-dimensional separation-preserving projection of the input data. Our method gives a non-spherical analog of the classical dimension reduction, based on singular value decomposition, that, among several other applications, forms a key component of the celebrated spherical clustering algorithm of Vempala and Wang [VW04]. As applications, we obtain an algorithm to (1) cluster an arbitrary total-variation separated mixture of $k$ centered (i.e., zero-mean) Gaussians with $n\geq \operatorname{poly}(d) f(w_{\min}^{-1})$ samples and $\operatorname{poly}(n)$ time, and (2) cluster an arbitrary total-variation separated mixture of $k$ Gaussians with identical but arbitrary unknown covariance with $n \geq d^{O(\log w_{\min}^{-1})} f(w_{\min}^{-1})$ samples and $n^{O(\log w_{\min}^{-1})}$ time. Here, $w_{\min}$ is the minimum mixing weight of the input mixture, and $f$ does not depend on the dimension $d$. Our algorithms naturally extend to tolerating a dimension-independent fraction of arbitrary outliers. Before this work, the techniques in the state-of-the-art non-spherical clustering algorithms needed $d^{O(k)} f(w_{\min}^{-1})$ samples and time for clustering such mixtures. Our results may come as a surprise in the context of the $d^{Ω(k)}$ statistical query and sum-of-squares lower bounds [DKS17, DKPP24] for clustering non-spherical Gaussian mixtures. While these results are usually thought to rule out $d^{o(k)}$ cost algorithms for the problem, our results show that the lower bounds can in fact be circumvented for a remarkably general class of Gaussian mixtures.

聚类算法高斯混合降维平方和

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。