用改进的散度和无限聚类模型,让深度聚类自动确定最佳簇数。
Deep Clustering using Dirichlet Process Gaussian Mixture and Alpha Jensen-Shannon Divergence Clustering Loss
- 采用闭式Jensen-Shannon散度替代传统不对称损失,提升聚类稳定性。
- 引入狄利克雷过程高斯混合模型,实现潜在空间中聚类数的自适应调整。
- 无需预先设定簇数,适用于大规模数据集,如MIT67和CIFAR100。
深度聚类是深度学习中的新兴方向,将传统聚类应用于深度特征空间。然而,聚类与深度学习常难以兼顾。在基于自编码器的深度聚类中,关键挑战在于如何联合优化聚类与降维,使隐藏层权重不仅受重建损失引导,也受聚类损失函数影响。现有最优方法存在两大缺陷:其一,聚类损失依赖对称性差的KL散度;其二,假设需事先知晓数据集的聚类数量。本文针对上述问题进行改进:首先使用闭式形式的Jensen-Shannon散度解决非对称问题;其次引入狄利克雷过程高斯混合模型(DPGMM),实现潜在空间中聚类与模型选择的联合优化,称为深度模型选择。聚类数量不固定,随训练过程动态逼近最优值,无需先验知识。我们在包含大量类别(如MIT67、CIFAR100)的数据集上评估该方法,并与传统贝叶斯变分模型及深度聚类方法对比,结果令人信服。
原文摘要 · Abstract (English)
Deep clustering is an emerging topic in deep learning where traditional clustering is performed in deep learning feature space. However, clustering and deep learning are often mutually exclusive. In the autoencoder based deep clustering, the challenge is how to jointly optimize both clustering and dimension reduction together, so that the weights in the hidden layers are not only guided by reconstruction loss, but also by a loss function associated with clustering. The current state-of-the-art has two fundamental flaws. First, they rely on the mathematical convenience of Kullback-Leibler divergence for the clustering loss function but the former is asymmetric. Secondly, they assume the prior knowledge on the number of clusters is always available for their dataset of interest. This paper tries to improve on these problems. In the first problem, we use a Jensen-Shannon divergence to overcome the asymmetric issue, specifically using a closed form variant. Next, we introduce an infinite cluster representation using Dirichlet process Gaussian mixture model for joint clustering and model selection in the latent space which we called deep model selection. The number of clusters in the latent space are not fixed but instead vary accordingly as they gradually approach the optimal number during training. Thus, prior knowledge is not required. We evaluate our proposed deep model selection method with traditional model selection on large class number datasets such as MIT67 and CIFAR100 and also compare with both traditional variational Bayes model and deep clustering method with convincing results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。