Chameleon2++通过近似搜索与多级分割,实现大规模数据聚类的高效高质处理。
Chameleon2++: An Efficient and Scalable Variant Of Chameleon Clustering
- 用Annoy做近似k-NN,加速图生成;
- 改用hMETIS多级分割,提升分区质量;
- 保留洪水填充合并策略,保证聚类连通性。
层次聚类在大数据挖掘中仍是基础挑战,传统方法难以有效扩展。现有基于Chameleon的算法(如Chameleon2、M-Chameleon、INNGS-Chameleon)仍存在O(n²)计算复杂度问题。本文在Chameleon2基础上提出Chameleon2++:首先,采用Annoy进行近似k-NN搜索,显著降低图生成时间;其次,改用hMETIS多级分割算法替代递归二分法,对图生成中的近似更鲁棒,且配置少;最后,保留洪水填充启发式,确保分区连通性与平衡性,并优化合并策略。该改进使整体复杂度降至O(n log n),在先前研究使用的基准数据集上,聚类质量平均提升4%,验证了大规模层次聚类中效率与质量可兼得。
原文摘要 · Abstract (English)
Hierarchical clustering remains a fundamental challenge in data mining, particularly when dealing with large-scale datasets where traditional approaches fail to scale effectively. Recent Chameleon-based algorithms - Chameleon2, M-Chameleon, and INNGS-Chameleon have proposed advanced strategies but they still suffer from $O(n^2)$ computational complexity, especially for large datasets. With Chameleon2 as the base algorithm, we introduce Chameleon2++ that addresses this challenge. Our algorithm has three parts. First, Graph Generation - we propose an approximate $k$-NN search instead of an exact one, specifically we integrate with the Annoy algorithm. This results in fast approximate nearest neighbor computation, significantly reducing the graph generation time. Second, Graph Partitioning - we propose use of a multi-level partitioning algorithm instead of a recursive bisection one. Specifically we adapt the hMETIS algorithm instead of the FM. This is because multi-level algorithms are robust to approximation introduced in the graph generation phase yielding higher-quality partitions, and that too with minimum configuration requirements. Third, Merging - we retain the flood fill heuristic that ensures connected balanced components in the partitions as well as efficient partition merging criteria leading to the final clusters. These enhancements reduce the overall time complexity to $O(n\log n)$, achieving scalability. On real-world benchmark datasets used in prior Chameleon works, Chameleon2++ delivers an average of 4% improvement in clustering quality. This demonstrates that algorithmic efficiency and clustering quality can co-exist in large-scale hierarchical clustering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。