将对比学习融入掩码特征重建,提升图自监督学习效果
CORE: Contrastive Masked Feature Reconstruction on Graphs
- 用原始与重构特征构建正样本对,强化上下文信息学习
- 以掩码节点为负样本,结合重建与判别优势
- 在节点和图分类任务上超越现有方法,最高提升3.76%
在图的自监督学习快速发展的背景下,生成式与对比式方法成为主流。本文聚焦掩码特征重建(MFR),一种通过自监督方式恢复被掩码节点原始特征的生成技术。我们发现,MFR与图对比学习(GCL)均旨在最大化相似元素间的一致性。基于此,揭示了一个新理论洞察:在特定条件下,MFR与节点级GCL的目标趋于一致,尽管其操作机制不同。这表明二者互补而非根本对立,促使我们探索融合路径。为此提出对比掩码特征重建(CORE)框架,将对比学习引入MFR。具体地,仅在原始特征与重构特征之间形成正样本对,引导编码器优先关注上下文信息而非节点自身特征;同时利用掩码节点作为负样本,结合MFR的重建能力与GCL的判别能力,更有效地捕捉图的内在结构。实验表明,所提框架在节点分类和图分类任务中显著优于MFR,达到领先水平。特别地,在节点分类任务上,相比GraphMAE和GraphMAE2分别提升最高2.80%和3.72%;在图分类任务上,分别提升最高3.82%和3.76%。
原文摘要 · Abstract (English)
In the rapidly evolving field of self-supervised learning on graphs, generative and contrastive methodologies have emerged as two dominant approaches. Our study focuses on masked feature reconstruction (MFR), a generative technique where a model learns to restore the raw features of masked nodes in a self-supervised manner. We observe that both MFR and graph contrastive learning (GCL) aim to maximize agreement between similar elements. Building on this observation, we reveal a novel theoretical insight: under specific conditions, the objectives of MFR and node-level GCL converge, despite their distinct operational mechanisms. This theoretical connection suggests these approaches are complementary rather than fundamentally different, prompting us to explore their integration to enhance self-supervised learning on graphs. Our research presents Contrastive Masked Feature Reconstruction (CORE), a novel graph self-supervised learning framework that integrates contrastive learning into MFR. Specifically, we form positive pairs exclusively between the original and reconstructed features of masked nodes, encouraging the encoder to prioritize contextual information over the node's own features. Additionally, we leverage the masked nodes themselves as negative samples, combining MFR's reconstructive power with GCL's discriminative ability to better capture intrinsic graph structures. Empirically, our proposed framework CORE significantly outperforms MFR across node and graph classification tasks, demonstrating state-of-the-art results. In particular, CORE surpasses GraphMAE and GraphMAE2 by up to 2.80% and 3.72% on node classification tasks, and by up to 3.82% and 3.76% on graph classification tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。