arXiv:2503.22241cs.AI2025-03被引 3

用多模态大模型做个性化聚类,让结果更贴合用户偏好。

Agent-Centric Personalized Multiple Clustering with Multi-Modal LLMs

  • 以多模态大模型为代理,在关系图中搜索用户兴趣对应的聚类。
  • 在卡牌排序和花色任务上分别达到0.9667和0.9481的NMI分数。
  • 通过兴趣导向嵌入过滤冗余边,提升搜索效率,适合需要个性化的场景。

个性化多聚类旨在根据不同的用户特定方面生成数据集的多种划分,而非单一聚类。该任务近年来因能适应用户偏好差异而受到关注。现有方法主要使用CLIP嵌入结合代理学习提取偏向用户聚类偏好的表示,但CLIP侧重粗粒度图文对齐,缺乏对用户兴趣的深层语境理解。为此,我们提出一种以代理为中心的个性化聚类框架,利用多模态大语言模型(MLLMs)作为代理,通过全面遍历关系图来基于用户兴趣搜索聚类。得益于MLLMs的先进推理能力,所获聚类更贴近用户定义的标准。为降低计算开销,我们使用MLLMs提取的兴趣导向嵌入构建关系图,并依据嵌入相似性过滤大量弱连接边,从而实现高效代理遍历。实验结果表明,该方法在Card Order和Card Suits基准上分别取得0.9667和0.9481的NMI分数,相较SOTA模型提升超过140%。

原文摘要 · Abstract (English)

Personalized multiple clustering aims to generate diverse partitions of a dataset based on different user-specific aspects, rather than a single clustering. It has recently drawn research interest for accommodating varying user preferences. Recent approaches primarily use CLIP embeddings with proxy learning to extract representations biased toward user clustering preferences. However, CLIP primarily focuses on coarse image-text alignment, lacking a deep contextual understanding of user interests. To overcome these limitations, we propose an agent-centric personalized clustering framework that leverages multi-modal large language models (MLLMs) as agents to comprehensively traverse a relational graph to search for clusters based on user interests. Due to the advanced reasoning mechanism of MLLMs, the obtained clusters align more closely with user-defined criteria than those obtained from CLIP-based representations. To reduce computational overhead, we shorten the agents' traversal path by constructing a relational graph using user-interest-biased embeddings extracted by MLLMs. A large number of weakly connected edges can be filtered out based on embedding similarity, facilitating an efficient traversal search for agents. Experimental results show that the proposed method achieves NMI scores of 0.9667 and 0.9481 on the Card Order and Card Suits benchmarks, respectively, largely improving the SOTA model by over 140%.

个性化聚类多模态大模型关系图搜索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。