arXiv:2410.18130cs.LGcs.CL2024-10被引 10

通过聚类精修负采样,解决文本分类中过聚类问题。

Graph Contrastive Learning via Cluster-refined Negative Sampling for Semi-supervised Text Classification

  • 基于BERT与图神经网络学习文本表征,用聚类生成伪标签。
  • 负样本从不同聚类中选取,避免相似节点被误判为负对。
  • 自校正机制保留远距离但真实负样本,适合半监督场景。

图对比学习(GCL)因能从无标签数据中生成自监督信号,广泛应用于文本分类任务。然而,现有基于GCL的方法常受负采样偏差影响,导致相似节点被错误配对为负样本,引发过聚类问题(同一类样本被分到不同簇)。为此,本文提出一种新方法ClusterText,结合预训练模型BERT与图神经网络学习文本表示,并引入聚类精修策略:通过聚类获得伪标签,每个节点的负样本从其他簇中采样。此外,设计自校正机制,通过计算同簇内节点间的欧氏距离,仍可将远距离节点选为负样本,缓解因聚类不一致造成的真负样本丢失。实验表明,ClusterText在文本分类任务中表现优越,具备良好的可扩展性,能有效从大量数据中提取关键信息。

原文摘要 · Abstract (English)

Graph contrastive learning (GCL) has been widely applied to text classification tasks due to its ability to generate self-supervised signals from unlabeled data, thus facilitating model training. However, existing GCL-based text classification methods often suffer from negative sampling bias, where similar nodes are incorrectly paired as negative pairs. This can lead to over-clustering, where instances of the same class are divided into different clusters. To address the over-clustering issue, we propose an innovative GCL-based method of graph contrastive learning via cluster-refined negative sampling for semi-supervised text classification, namely ClusterText. Firstly, we combine the pre-trained model Bert with graph neural networks to learn text representations. Secondly, we introduce a clustering refinement strategy, which clusters the learned text representations to obtain pseudo labels. For each text node, its negative sample set is drawn from different clusters. Additionally, we propose a self-correction mechanism to mitigate the loss of true negative samples caused by clustering inconsistency. By calculating the Euclidean distance between each text node and other nodes within the same cluster, distant nodes are still selected as negative samples. Our proposed ClusterText demonstrates good scalable computing, as it can effectively extract important information from from a large amount of data. Experimental results demonstrate the superiority of ClusterText in text classification tasks.

图对比学习半监督文本分类聚类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。