提出首个小批量核k-means算法,速度提升10-100倍且精度损失小。
Mini-Batch Kernel $k$-means
- 采用小批量采样加速核k-means,单次迭代仅需$ ilde{O}(kb^2)$时间。
- 实验显示在保持质量的前提下实现10-100倍提速,显著改善运行效率。
- 理论证明在合理初始化下可快速收敛,适合大规模数据聚类场景。
我们提出了首个小批量核k-means算法,相比全批量算法实现数量级的运行时间提升。单次迭代耗时$ ilde{O}(kb^2)$,远快于全批量核k-means所需的$O(n^2)$,其中$n$为数据集大小,$b$为批大小。大量实验表明,该算法在几乎不损失精度的情况下持续实现10-100倍加速,解决了核k-means因运行缓慢而难以实际应用的问题。我们进一步在早期终止条件下提供理论分析,证明当批大小为$ ilde{igOmega}( ext{max}ackslash{γ^4, γ^2ackslash} imes ε^{-2})$时,算法以高概率在$O(γ^2/ε)$次迭代内终止,其中$γ$为特征空间中点的范数上界,$ε$为终止阈值。该分析适用于任意合理中心初始化;若使用$k$-means++初始化,则期望近似比为$O( ext{log} k)$。对于归一化核(如高斯或拉普拉斯核),有$γ=1$。取$ε=O(1)$且$b=Θ( ext{log} n)$,算法可在$O(1)$次迭代内终止,每次迭代耗时$ ilde{O}(k)$。
原文摘要 · Abstract (English)
We present the first mini-batch kernel $k$-means algorithm, offering an order of magnitude improvement in running time compared to the full batch algorithm. A single iteration of our algorithm takes $\widetilde{O}(kb^2)$ time, significantly faster than the $O(n^2)$ time required by the full batch kernel $k$-means, where $n$ is the dataset size and $b$ is the batch size. Extensive experiments demonstrate that our algorithm consistently achieves a 10-100x speedup with minimal loss in quality, addressing the slow runtime that has limited kernel $k$-means adoption in practice. We further complement these results with a theoretical analysis under an early stopping condition, proving that with a batch size of $\widetildeΩ(\max \{γ^{4}, γ^{2}\} \cdot ε^{-2})$, the algorithm terminates in $O(γ^2/ε)$ iterations with high probability, where $γ$ bounds the norm of points in feature space and $ε$ is a termination threshold. Our analysis holds for any reasonable center initialization, and when using $k$-means++ initialization, the algorithm achieves an approximation ratio of $O(\log k)$ in expectation. For normalized kernels, such as Gaussian or Laplacian it holds that $γ=1$. Taking $ε= O(1)$ and $b=Θ(\log n)$, the algorithm terminates in $O(1)$ iterations, with each iteration running in $\widetilde{O}(k)$ time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。