用注意力优化运输生成可靠伪标签,提升短文本聚类效果
Reliable Pseudo-labeling via Optimal Transport with Attention for Short Text Clustering
- 通过注意力机制捕捉样本语义关系,融入最优传输求解伪标签
- 在多个数据集上显著超越现有方法,最高提升6.2个点
- 适合处理类别不平衡的短文本聚类任务
短文本聚类在数据挖掘领域受到广泛关注。然而,短文本中包含的有效信息有限,常导致表征不具区分性,增加聚类难度。本文提出一种新型短文本聚类框架POTA(Reliable Pseudo-labeling via Optimal Transport with Attention),通过生成可靠的伪标签来辅助判别性表征学习。具体而言,POTA首先引入实例级注意力机制以捕捉样本间的语义关系,并将其作为语义一致性正则项融入最优传输问题。求解该最优传输问题可获得同时考虑样本间语义一致性和样本到聚类全局结构信息的可靠伪标签。此外,所提出的最优传输能自适应估计聚类分布,使POTA适用于不同程度的不平衡数据集。随后,利用伪标签引导对比学习生成判别性表征并实现高效聚类。大量实验表明,POTA优于当前最优方法。代码已公开于:https://github.com/YZH0905/POTA-STC/tree/main。
原文摘要 · Abstract (English)
Short text clustering has gained significant attention in the data mining community. However, the limited valuable information contained in short texts often leads to low-discriminative representations, increasing the difficulty of clustering. This paper proposes a novel short text clustering framework, called Reliable \textbf{P}seudo-labeling via \textbf{O}ptimal \textbf{T}ransport with \textbf{A}ttention for Short Text Clustering (\textbf{POTA}), that generate reliable pseudo-labels to aid discriminative representation learning for clustering. Specially, \textbf{POTA} first implements an instance-level attention mechanism to capture the semantic relationships among samples, which are then incorporated as a semantic consistency regularization term into an optimal transport problem. By solving this OT problem, we can yield reliable pseudo-labels that simultaneously account for sample-to-sample semantic consistency and sample-to-cluster global structure information. Additionally, the proposed OT can adaptively estimate cluster distributions, making \textbf{POTA} well-suited for varying degrees of imbalanced datasets. Then, we utilize the pseudo-labels to guide contrastive learning to generate discriminative representations and achieve efficient clustering. Extensive experiments demonstrate \textbf{POTA} outperforms state-of-the-art methods. The code is available at: \href{https://github.com/YZH0905/POTA-STC/tree/main}{https://github.com/YZH0905/POTA-STC/tree/main}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。