arXiv:2410.07858cs.LGcs.AI2024-10ICML被引 5

用预训练模型的输出直接构建层次聚类,效果更好且无需微调。

From Logits to Hierarchies: Hierarchical Clustering made Simple

  • 基于预训练聚类模型的logits构建层次结构,不需额外训练。
  • 在真实数据集上性能超越专用深度层次聚类模型。
  • 适用于任意输出logits的聚类模型,通用性强。

许多现实世界数据本身就具有层次结构,建模这种层次关系在无监督和有监督学习中都至关重要。尽管近期已出现专为层次聚类设计的深度架构,但本文对其提出批判性审视:这些方法在真实数据集上存在显著的可扩展性和性能瓶颈。为此,我们提出一种轻量级替代方案,利用预训练的非层次聚类模型输出的logits构建层次结构。结果表明,该方法在性能上优于专门的深度层次聚类模型,并可广泛应用于任意输出logits的预训练聚类模型,无需任何微调。为验证其通用性,我们还将该方法拓展至有监督场景,成功从预训练的ImageNet分类器中恢复出有意义的层次结构。实验显示,该方法在效率、可扩展性和性能方面均具备显著优势。

原文摘要 · Abstract (English)

The hierarchical structure inherent in many real-world datasets makes the modeling of such hierarchies a crucial objective in both unsupervised and supervised machine learning. While recent advancements have introduced deep architectures specifically designed for hierarchical clustering, we adopt a critical perspective on this line of research. Our findings reveal that these methods face significant limitations in scalability and performance when applied to realistic datasets. Given these findings, we present an alternative approach and introduce a lightweight method that builds on pre-trained non-hierarchical clustering models. Remarkably, our approach outperforms specialized deep models for hierarchical clustering, and it is broadly applicable to any pre-trained clustering model that outputs logits, without requiring any fine-tuning. To highlight the generality of our approach, we extend its application to a supervised setting, demonstrating its ability to recover meaningful hierarchies from a pre-trained ImageNet classifier. Our results establish a practical and effective alternative to existing deep hierarchical clustering methods, with significant advantages in efficiency, scalability and performance.

层次聚类预训练模型无监督学习高效算法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。