arXiv:2409.14455cs.DScs.GT2024-09

用稳定匹配法加速聚类评估,大数量簇场景下更快更准。

A High-Performance External Validity Index for Clustering with a Large Number of Clusters

  • 基于稳定匹配框架配对不同聚类结果,降低计算复杂度。
  • 相比传统方法提速至O(N²),在真实与合成数据上精度相当。
  • 适合大规模、多簇数据,可轻松集成到PyTorch/TensorFlow中。

本文提出一种高效率的外部有效性评估指标——基于稳定匹配的配对算法(SMBP),适用于大规模数据集和大量聚类的情况。SMBP利用稳定匹配框架对不同聚类方法的结果进行配对,将计算复杂度从传统最大权重匹配(MWM)的O(N³)降至O(N²),显著提升效率。在真实世界与合成数据上的全面评估表明,SMBP在准确率上与MWM相当,且在平衡、非平衡及大规模数据上均表现优异。该算法易于在PyTorch和TensorFlow等机器学习框架中实现,为现代聚类任务提供了一种可扩展、实用的解决方案。实验验证其可作为最大匹配度量(MMM)和中心点比值(CR)等现有方法的有效替代。

原文摘要 · Abstract (English)

This paper introduces the Stable Matching Based Pairing (SMBP) algorithm, a high-performance external validity index for clustering evaluation in large-scale datasets with a large number of clusters. SMBP leverages the stable matching framework to pair clusters across different clustering methods, significantly reducing computational complexity to $O(N^2)$, compared to traditional Maximum Weighted Matching (MWM) with $O(N^3)$ complexity. Through comprehensive evaluations on real-world and synthetic datasets, SMBP demonstrates comparable accuracy to MWM and superior computational efficiency. It is particularly effective for balanced, unbalanced, and large-scale datasets with a large number of clusters, making it a scalable and practical solution for modern clustering tasks. Additionally, SMBP is easily implementable within machine learning frameworks like PyTorch and TensorFlow, offering a robust tool for big data applications. The algorithm is validated through extensive experiments, showcasing its potential as a powerful alternative to existing methods such as Maximum Match Measure (MMM) and Centroid Ratio (CR).

聚类评估稳定匹配高效算法大数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。