用语义引导对比学习,让InfoNCE自动发现正样本,性能提升超9%。
InfoNCE is a Free Lunch for Semantically guided Graph Contrastive Learning
- 将对比学习视为语义引导的正/未标记学习,重新定义正样本
- 利用InfoNCE特性自动识别语义相似节点对,构建更优对比任务
- 在跨域和同分布场景均显著提升,适合图预训练与图LLM应用
图对比学习(GCL)作为重要的图预训练方法,在图基础模型或图增强大模型研究中持续发挥关键作用。传统GCL通过数据增强构造自监督任务,将增强对视为正样本,其余为负样本,但导致语义相近的样本被误判为负,造成严重采样偏差。本文提出将GCL视为正/未标记(PU)学习问题,主张以语义相似性定义正样本,其余未知语义样本视为未标记。为此,提出IFL-GCL方法,利用InfoNCE作为“免费午餐”提取语义信息。理论证明:在InfoNCE下,节点表示相似度与对应对比样本为正的概率一致。据此修正样本并重构最大似然目标,得到新形式的InfoNCE损失。大量实验表明,IFL-GCL在图预训练框架及图增强大模型中均有显著提升,跨域与同分布场景下最高达9.05%增益,验证了语义引导的有效性。代码已开源:https://github.com/Camel-Prince/IFL-GCL。
原文摘要 · Abstract (English)
As an important graph pre-training method, Graph Contrastive Learning (GCL) continues to play a crucial role in the ongoing surge of research on graph foundation models or LLM as enhancer for graphs. Traditional GCL optimizes InfoNCE by using augmentations to define self-supervised tasks, treating augmented pairs as positive samples and others as negative. However, this leads to semantically similar pairs being classified as negative, causing significant sampling bias and limiting performance. In this paper, we argue that GCL is essentially a Positive-Unlabeled (PU) learning problem, where the definition of self-supervised tasks should be semantically guided, i.e., augmented samples with similar semantics are considered positive, while others, with unknown semantics, are treated as unlabeled. From this perspective, the key lies in how to extract semantic information. To achieve this, we propose IFL-GCL, using InfoNCE as a "free lunch" to extract semantic information. Specifically, We first prove that under InfoNCE, the representation similarity of node pairs aligns with the probability that the corresponding contrastive sample is positive. Then we redefine the maximum likelihood objective based on the corrected samples, leading to a new InfoNCE loss function. Extensive experiments on both the graph pretraining framework and LLM as an enhancer show significantly improvements of IFL-GCL in both IID and OOD scenarios, achieving up to a 9.05% improvement, validating the effectiveness of semantically guided. Code for IFL-GCL is publicly available at: https://github.com/Camel-Prince/IFL-GCL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。