arXiv:2607.24537cs.LGcs.NE2026-07

K-SCAN融合量化与密度分析,实现百万级数据的快速非线性聚类。

The K-SCAN Clustering Algorithm

论文配图:The K-SCAN Clustering Algorithm
图 1 · 摘自论文原文
  • 先用随机小批量K-Means压缩数据为加权微簇,再做密度分析。
  • 处理10^6样本仅需线性时间,比BIRCH快3倍以上,噪声达55%仍稳定。
  • 适合大规模高维数据聚类,尤其擅长捕捉非球形结构。

在大数据时代,聚类算法的可扩展性构成关键挑战。传统基于密度的方法(如DBSCAN)虽对噪声鲁棒且能发现非线性聚类,但其二次时间复杂度 $O(N^2)$ 严重限制了应用范围。相反,划分型算法(如K-Means)虽具线性复杂度 $O(N)$,却强制球形分组,对异常值敏感。本文提出K-SCAN——一种新型混合算法,优化该权衡。方法先通过预向量量化(随机迷你批次K-Means)提取一组加权微簇,再进行后续密度基结构分析。在最大达 $10^6$ 样本的数据集上实证评估表明,所提方案具备线性计算复杂度。K-SCAN相比层次式BIRCH算法实现超3倍速度提升,避免了树结构管理的开销。该方法能精确识别非线性流形,即使在噪声占数据总量55%的情况下仍保持结构稳定(调整兰德指数 > 0.99)。主要局限在于对过平滑的敏感性以及难以分离局部密度差异极大的聚类,在复杂视觉空间中可能损失细微拓扑细节。

原文摘要 · Abstract (English)

In the Big Data era, the scalability of clustering algorithms constitutes a key challenge. Traditional density-based methods (e.g., DBSCAN) offer robustness to noise and the ability to detect non-linear clusters, yet their quadratic time complexity $O(N^2)$ drastically limits their applicability. Conversely, partitional algorithms (e.g., K-Means), with their linear complexity $O(N)$, impose sphericity on the resulting groups and fail in the presence of outliers. This paper presents K-SCAN -- a novel hybrid algorithm that optimizes this trade-off. The method integrates preliminary vector quantization (stochastic Mini-Batch K-Means) to extract a reduced set of weighted micro-clusters, followed by a subsequent density-based structural analysis. Empirical evaluation on datasets of up to $10^6$ samples confirms the linear computational complexity of the proposed solution. K-SCAN achieves more than a 3-fold speed-up over the hierarchical BIRCH algorithm, avoiding the costly management of tree-based structures. The method precisely identifies non-linear manifolds while maintaining structural stability (Adjusted Rand Index > 0.99), even with noise levels reaching 55\% of the data volume. The main limitation of the proposed algorithm, which could not be fully eliminated in the present study, remains its susceptibility to over-smoothing and its difficulty in separating clusters with highly heterogeneous local density. In complex visual spaces, this can lead to the loss of the finest topological details.

聚类算法大数据密度聚类加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。