用多模态大模型生成可解释的多聚类结果，支持用户自定义标准。
ESMC: MLLM-Based Embedding Selection for Explainable Multiple Clustering
- 利用MLLM文本隐藏状态提取特征嵌入，实现用户定制聚类。
- 在多个数据集上达到优于主流方法的聚类精度。
- 适合需要灵活、可解释聚类结果的研究者和应用开发
传统深度聚类方法虽取得显著进展，但每份数据集仅能提供单一聚类结果。这一局限源于其对固定数据分布的假设，难以满足用户需求并导致聚类效果不佳。本文研究如何利用多模态大语言模型（MLLM）实现用户驱动的聚类，强调其对用户指定语义要求的适应性。然而，直接使用MLLM输出进行聚类存在生成非结构化、泛化描述的风险。为此，我们发现MLLM中文本标记的隐藏状态与对应特征密切相关，并据此构建聚类嵌入。同时引入轻量级聚类头结合伪标签学习，显著提升聚类准确率。大量实验表明，该方法在多种数据集和评估指标上表现优异。
原文摘要 · Abstract (English)
Typical deep clustering methods, while achieving notable progress, can only provide one clustering result per dataset. This limitation arises from their assumption of a fixed underlying data distribution, which may fail to meet user needs and provide unsatisfactory clustering outcomes. Our work investigates how multi-modal large language models (MLLMs) can be leveraged to achieve user-driven clustering, emphasizing their adaptability to user-specified semantic requirements. However, directly using MLLM output for clustering has risks for producing unstructured and generic image descriptions instead of feature-specific and concrete ones. To address these issues, our method first discovers that MLLMs' hidden states of text tokens are strongly related to the corresponding features, and leverages these embeddings to perform clusterings from any user-defined criteria. We also employ a lightweight clustering head augmented with pseudo-label learning, significantly enhancing clustering accuracy. Extensive experiments demonstrate its competitive performance on diverse datasets and metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。