让嵌入向量更贴合数据中的簇结构,提升聚类与去噪效果。
Cluster Specific Representation Learning
- 学习每个簇专属的嵌入表示,不依赖下游任务
- 在自编码器、对比学习等框架中均提升聚类效果
- 适合需挖掘潜在簇结构的任务,如异常检测
表示学习旨在从数据中提取有意义的低维嵌入。尽管应用广泛,但对‘优质’表示尚无明确定义。通常通过聚类、去噪等下游任务表现评估,但此方法存在局限:某任务表现优的表示未必适用于其他任务。为此,本文提出一种下游无关的表示学习范式:当数据存在固有簇时,表示应具有簇特异性。我们设计了一种元算法,联合学习簇特异性表示与簇分配。该方法可无缝集成至自编码器、变分自编码器、对比学习模型及受限玻尔兹曼机等框架。实验表明,相比标准嵌入,其能更有效捕捉数据内在簇结构,在去噪和聚类任务中性能更优。虽略微增加运行时间和参数量,但提升显著。
原文摘要 · Abstract (English)
Representation learning aims to extract meaningful lower-dimensional embeddings from data, known as representations. Despite its widespread application, there is no established definition of a ``good'' representation. Typically, the representation quality is evaluated based on its performance in downstream tasks such as clustering, de-noising, etc. However, this task-specific approach has a limitation where a representation that performs well for one task may not necessarily be effective for another. This highlights the need for a more agnostic formulation, which is the focus of our work. We propose a downstream-agnostic formulation: when inherent clusters exist in the data, the representations should be specific to each cluster. Under this idea, we develop a meta-algorithm that jointly learns cluster-specific representations and cluster assignments. As our approach is easy to integrate with any representation learning framework, we demonstrate its effectiveness in various setups, including Autoencoders, Variational Autoencoders, Contrastive learning models, and Restricted Boltzmann Machines. We qualitatively compare our cluster-specific embeddings to standard embeddings and downstream tasks such as de-noising and clustering. While our method slightly increases runtime and parameters compared to the standard model, the experiments clearly show that it extracts the inherent cluster structures in the data, resulting in improved performance in relevant applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。