arXiv:2410.10365cs.LGcs.AI2024-10被引 28

不依赖正样本的图对比学习,利用高频信息加速收敛

SpeGCL: Self-supervised Graph Spectrum Contrastive Learning without Positive Samples

  • 在傅里叶空间构建对比学习,分离并利用高/低频特征
  • 仅用负样本优化,理论证明比拉近正样本对更有效
  • 在无监督、半监督等场景均超越现有方法

图对比学习(GCL)在应对输入数据噪声和波动方面表现优异,广泛应用于社交网络、知识图谱等领域。研究发现,增强图之间的高频信息差异大于低频信息差异。然而,现有GCL方法主要关注时域(低频信息)的节点特征表示,未能有效利用高频信息以加速模型收敛。此外,传统GCL通过拉近正样本对距离、推远正负样本对距离来优化图嵌入表示,但理论分析表明,图对比学习更受益于推远负样本对距离,而非拉近正样本对。为此,本文提出一种无需正样本的新型谱域图对比学习框架SpeGCL。具体而言,为解决现有方法无法利用高频信息的问题,SpeGCL采用傅里叶变换提取节点特征的高低频成分,并在傅里叶空间构建对比学习机制,以获得更优的节点特征表示。同时,SpeGCL完全依赖负样本进行图嵌入优化,并提供了使用负样本的理论依据。在无监督学习、迁移学习和半监督学习任务上的大量实验验证了其优于当前最先进GCL方法的性能。

原文摘要 · Abstract (English)

Graph Contrastive Learning (GCL) excels at managing noise and fluctuations in input data, making it popular in various fields (e.g., social networks, and knowledge graphs). Our study finds that the difference in high-frequency information between augmented graphs is greater than that in low-frequency information. However, most existing GCL methods focus mainly on the time domain (low-frequency information) for node feature representations and cannot make good use of high-frequency information to speed up model convergence. Furthermore, existing GCL paradigms optimize graph embedding representations by pulling the distance between positive sample pairs closer and pushing the distance between positive and negative sample pairs farther away, but our theoretical analysis shows that graph contrastive learning benefits from pushing negative pairs farther away rather than pulling positive pairs closer. To solve the above-mentioned problems, we propose a novel spectral GCL framework without positive samples, named SpeGCL. Specifically, to solve the problem that existing GCL methods cannot utilize high-frequency information, SpeGCL uses a Fourier transform to extract high-frequency and low-frequency information of node features, and constructs a contrastive learning mechanism in a Fourier space to obtain better node feature representation. Furthermore, SpeGCL relies entirely on negative samples to refine the graph embedding. We also provide a theoretical justification for the efficacy of using only negative samples in SpeGCL. Extensive experiments on un-supervised learning, transfer learning, and semi-supervised learning have validated the superiority of our SpeGCL framework over the state-of-the-art GCL methods.

图神经网络对比学习自监督谱方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。