arXiv:2503.17908cs.LG2025-03AAAI被引 16

提出少而精的负样本策略,让图对比学习更高效

Does GCL Need a Large Number of Negative Samples? Enhancing Graph Contrastive Learning with Effective and Efficient Negative Sampling

  • 用少量高质量负样本替代海量随机负样本
  • 在多个数据集上达到更好性能且训练快一倍以上
  • 适合资源有限或追求高效训练的研究者

图对比学习(GCL)通过自监督方式学习低维图表示,主要依赖实例判别任务,即从图中人工构建正负样本对,提升正样本相似度、降低负样本相似度。受其他领域对比学习成功启发,学界普遍认为大量负样本是GCL有效性的关键。因此,尽管计算开销巨大,多数GCL方法仍尽可能使用大量负节点对以提升性能。然而,由于图中节点相互关联,我们质疑:将更多负节点纳入是否真能提升模型表现?本文研究了常用InfoNCE损失中负节点的作用,发现:(1) 反直觉地,过多负节点反而削弱模型区分语义不同节点的能力;(2) 少量高质量、非拓扑耦合的负节点即可显著增强表示可区分性。基于此,我们提出E2Neg方法,仅需极少量代表性负样本即可学习判别性表示,大幅降低计算开销并加速训练。实验表明,相比其他GCL方法,E2Neg在多个数据集上兼具更高效率与更强性能。

原文摘要 · Abstract (English)

Graph Contrastive Learning (GCL) aims to self-supervised learn low-dimensional graph representations, primarily through instance discrimination, which involves manually mining positive and negative pairs from graphs, increasing the similarity of positive pairs while decreasing negative pairs. Drawing from the success of Contrastive Learning (CL) in other domains, a consensus has been reached that the effectiveness of GCLs depends on a large number of negative pairs. As a result, despite the significant computational overhead, GCLs typically leverage as many negative node pairs as possible to improve model performance. However, given that nodes within a graph are interconnected, we argue that nodes cannot be treated as independent instances. Therefore, we challenge this consensus: Does employing more negative nodes lead to a more effective GCL model? To answer this, we explore the role of negative nodes in the commonly used InfoNCE loss for GCL and observe that: (1) Counterintuitively, a large number of negative nodes can actually hinder the model's ability to distinguish nodes with different semantics. (2) A smaller number of high-quality and non-topologically coupled negative nodes are sufficient to enhance the discriminability of representations. Based on these findings, we propose a new method called GCL with Effective and Efficient Negative samples, E2Neg, which learns discriminative representations using only a very small set of representative negative samples. E2Neg significantly reduces computational overhead and speeds up model training. We demonstrate the effectiveness and efficiency of E2Neg across multiple datasets compared to other GCL methods.

图神经网络对比学习负样本采样高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。