用稳定匹配法加速聚类评估,大数量簇场景下更快更准。
A High-Performance External Validity Index for Clustering with a Large Number of Clusters
- 基于稳定匹配框架配对不同聚类结果,降低计算复杂度。
- 相比传统方法提速至O(N²),在真实与合成数据上精度相当。
- 适合大规模、多簇数据,可轻松集成到PyTorch/TensorFlow中。
本文提出一种高效率的外部有效性评估指标——基于稳定匹配的配对算法(SMBP),适用于大规模数据集和大量聚类的情况。SMBP利用稳定匹配框架对不同聚类方法的结果进行配对,将计算复杂度从传统最大权重匹配(MWM)的O(N³)降至O(N²),显著提升效率。在真实世界与合成数据上的全面评估表明,SMBP在准确率上与MWM相当,且在平衡、非平衡及大规模数据上均表现优异。该算法易于在PyTorch和TensorFlow等机器学习框架中实现,为现代聚类任务提供了一种可扩展、实用的解决方案。实验验证其可作为最大匹配度量(MMM)和中心点比值(CR)等现有方法的有效替代。
原文摘要 · Abstract (English)
This paper introduces the Stable Matching Based Pairing (SMBP) algorithm, a high-performance external validity index for clustering evaluation in large-scale datasets with a large number of clusters. SMBP leverages the stable matching framework to pair clusters across different clustering methods, significantly reducing computational complexity to $O(N^2)$, compared to traditional Maximum Weighted Matching (MWM) with $O(N^3)$ complexity. Through comprehensive evaluations on real-world and synthetic datasets, SMBP demonstrates comparable accuracy to MWM and superior computational efficiency. It is particularly effective for balanced, unbalanced, and large-scale datasets with a large number of clusters, making it a scalable and practical solution for modern clustering tasks. Additionally, SMBP is easily implementable within machine learning frameworks like PyTorch and TensorFlow, offering a robust tool for big data applications. The algorithm is validated through extensive experiments, showcasing its potential as a powerful alternative to existing methods such as Maximum Match Measure (MMM) and Centroid Ratio (CR).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。