提出新型聚类评估指标CDL,更好识别非凸与不规则数据的真实聚类数。
Central Description Length (CDL) Clustering Validation Index

- 基于概率上界估计真实中心描述长度,融合紧凑性与中心偏移
- 在非凸、任意形状数据上比传统指标更准,准确率更高
- 无需核变换或额外调参,适用于图像嵌入聚类等场景
在无标签的传感器、图像或过程数据分析中,选择合适的聚类算法及其超参数是常见挑战。聚类验证指标(CVIs)可对候选聚类结果进行内部评分,但多数主流指标依赖欧氏紧凑性与分离性,倾向于偏好紧凑、凸形划分,在非凸、不规则或密度变化的数据上表现下降,通常需通过核变换或替代距离度量补救,代价是额外调参与计算开销。本文提出中央描述长度(CDL)聚类验证指标,利用观测到的簇内紧凑性、估计的簇中心和协方差,计算与不可观测的真实簇中心相关的描述长度上界。该上界将簇内紧凑性与中心位移合并为单一可计算量,适用于任意聚类算法生成的划分。实现仅依赖可观测量(数据、划分、估计中心、估计协方差),无需真实标签。在具有非凸与任意形状簇的合成基准测试中,CDL-CVI更频繁选出参考聚类数,且达到更高的调整兰德指数(ARI)值,无需额外核预处理阶段。在图像基准(MNIST、CIFAR-10、STL-10)上,从冻结的无监督嵌入聚类,CDL-CVI在各试验中返回的聚类数均接近参考类别数量,适用范围涵盖K均值、DBSCAN与谱聚类。
原文摘要 · Abstract (English)
Selecting a clustering algorithm and its hyperparameters without labels is a common difficulty in engineering machine learning pipelines that work with unsupervised analysis of sensor, image, or process data. Clustering validation indices (CVIs) provide internal scores for ranking candidate clusterings, but most popular CVIs are built from Euclidean compactness and separation terms and so tend to favour compact, convex partitions. Their performance is known to degrade on non convex, irregular, or variable density data, where kernel transformations or alternative distance measures are typically used at the cost of additional tuning and computation. This paper introduces the Central Description Length (CDL) clustering validation index. CDL uses the observed within cluster compactness, the estimated cluster centers, and the estimated cluster covariances to compute a probabilistic upper bound on the description length associated with the unobservable true cluster centers. The bound condenses intra cluster compactness and centroid displacement into a single computable quantity and is evaluated on the partition produced by any clustering algorithm. The implementation uses only observable quantities (the data, the partition, the estimated centers, and the estimated covariances) and does not use ground truth labels. On synthetic benchmarks with non convex and arbitrary shape clusters, CDL-CVI selected the reference number of clusters more often and reached higher Adjusted Rand Index (ARI) values than the conventional CVIs we tested, without an additional kernel preprocessing stage. On image benchmarks (MNIST, CIFAR-10, STL-10) clustered from frozen unsupervised embeddings, CDL-CVI returned cluster numbers close to the reference class counts across K-means, DBSCAN, and spectral clustering in the reported trials.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。