提出低敏感性层次k-means聚类算法,提升动态大数据下的稳定性。
Average Sensitivity of Hierarchical $k$-Median Clustering
- 设计新算法,降低删除随机点时聚类结果的平均变化
- 理论证明算法平均敏感度低且聚类质量高
- 适合对稳定性要求高的大规模动态数据场景
层次聚类是广泛应用的无监督学习方法,但在现代算法中,数据集通常规模大且动态变化。若聚类结果对数据微小扰动敏感,算法实用性将显著下降。本文研究层次k-均值聚类问题,该问题结合了层次聚类与中心点聚类的优势,在理论和实践上均有吸引力。通过测量删除一个随机数据点后输出的期望变化,分析算法的平均敏感度。提出一种高效的层次k-均值聚类算法,并理论证明其具有低平均敏感度和高聚类质量。同时发现,单链接聚类和CLNSS的确定性变体表现出高平均敏感度,稳定性较差。实验验证了所提算法在鲁棒性和有效性上的优势。
原文摘要 · Abstract (English)
Hierarchical clustering is a widely used method for unsupervised learning with numerous applications. However, in the application of modern algorithms, the datasets studied are usually large and dynamic. If the hierarchical clustering is sensitive to small perturbations of the dataset, the usability of the algorithm will be greatly reduced. In this paper, we focus on the hierarchical $k$ -median clustering problem, which bridges hierarchical and centroid-based clustering while offering theoretical appeal, practical utility, and improved interpretability. We analyze the average sensitivity of algorithms for this problem by measuring the expected change in the output when a random data point is deleted. We propose an efficient algorithm for hierarchical $k$-median clustering and theoretically prove its low average sensitivity and high clustering quality. Additionally, we show that single linkage clustering and a deterministic variant of the CLNSS algorithm exhibit high average sensitivity, making them less stable. Finally, we validate the robustness and effectiveness of our algorithm through experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。