arXiv:2412.15295cs.DScs.LG2024-12被引 3

1D聚类速度提升4500倍,还保证精度,适合大数据和量化任务。

Log-Time K-Means Clustering for 1D Data: Novel Approaches with Proof and Implementation

  • 利用排序数据与二分查找,优化k-means++初始化和迭代过程。
  • 在大规模数据上实现4500倍加速,且保持与scikit-learn相当的聚类质量。
  • 算法已开源并支持JIT优化,适合需要高效聚类的工程应用。

聚类是机器学习中的关键任务,其中k-means因简单高效被广泛应用。尽管一维聚类常见,现有方法常未能利用1D数据结构,导致效率低下。本文提出优化的k-means++初始化与Lloyd算法,结合排序数据、前缀和与二分查找,显著提升计算性能。主要贡献包括:(1) 一种优化的k-簇算法,实现贪心k-means++初始化的$O(l \cdot k^2 \cdot \log n)$复杂度和Lloyd算法的$O(i \cdot k \cdot \log n)$复杂度,其中$l$为局部尝试次数,$i$为迭代次数;(2) 基于二分查找的双簇算法,实现$O(\log n)$运行时间,并确定性收敛至Lloyd局部最优解。实验表明,该方法在大规模数据上相较scikit-learn提速超4500倍,同时保持基于组内平方和(WCSS)的聚类质量。此外,在大模型量化任务中实现300倍加速,凸显其在新兴场景中的实用价值。本研究连接理论与实践,提供高效且可靠的1D k-means算法,已实现于一个支持即时编译优化的开源Python库中。

原文摘要 · Abstract (English)

Clustering is a key task in machine learning, with $k$-means being widely used for its simplicity and effectiveness. While 1D clustering is common, existing methods often fail to exploit the structure of 1D data, leading to inefficiencies. This thesis introduces optimized algorithms for $k$-means++ initialization and Lloyd's algorithm, leveraging sorted data, prefix sums, and binary search for improved computational performance. The main contributions are: (1) an optimized $k$-cluster algorithm achieving $O(l \cdot k^2 \cdot \log n)$ complexity for greedy $k$-means++ initialization and $O(i \cdot k \cdot \log n)$ for Lloyd's algorithm, where $l$ is the number of greedy $k$-means++ local trials, and $i$ is the number of Lloyd's algorithm iterations, and (2) a binary search-based two-cluster algorithm, achieving $O(\log n)$ runtime with deterministic convergence to a Lloyd's algorithm local minimum. Benchmarks demonstrate over a 4500x speedup compared to scikit-learn for large datasets while maintaining clustering quality measured by within-cluster sum of squares (WCSS). Additionally, the algorithms achieve a 300x speedup in an LLM quantization task, highlighting their utility in emerging applications. This thesis bridges theory and practice for 1D $k$-means clustering, delivering efficient and sound algorithms implemented in a JIT-optimized open-source Python library.

聚类算法优化1D数据量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。