发现图对比学习中正样本作用被高估,提出新方法恢复其学习效能。
Revisiting Positive Samples in Graph Contrastive Learning: From the Perspective of Message Passing
- 从狄利克雷能量视角揭示消息传递使正样本优化失效
- 仅传播高能量特征以增强正样本信号,用低能量特征构建可靠采样矩阵
- 实验验证新方法显著提升图对比学习性能,适合图表示学习研究者
图对比学习(GCL)通过最大化正样本间相似性、最小化负样本间相似性来训练图编码器,已成为主流的图预训练范式。普遍认为正样本对学习至关重要,理想情况下可帮助编码器捕捉图数据的内在语义与模式。然而我们发现:即使不使用正样本,GCL仍能取得竞争力表现。这促使我们重新审视正样本在GCL中的基本机制。从狄利克雷能量角度出发,理论发现:消息传递作为图编码器的核心机制,会弱化正样本的相似性最大化,导致正样本无法有效提供学习信号。为此,我们提出SPGCL,以缓解消息传递带来的信号稀释问题并恢复正样本的学习有效性。具体而言,我们发现高狄利克雷能量特征有助于正样本提供有效学习信号,而低能量特征虽对学习信号贡献小,却有利于正样本的可靠构造。基于此,SPGCL仅传播高能量特征,并利用低能量特征构建概率矩阵以实现可靠正样本采样。大量实验证明了该方法的有效性。
原文摘要 · Abstract (English)
Graph Contrastive Learning (GCL), which trains graph encoders by maximizing similarity between positive samples and minimizing it between negative ones, has emerged as a mainstream graph pre-training paradigm. It is widely recognized that positive samples are essential in GCLs. Ideally, maximizing the similarity of positive samples enables graph encoders to capture intrinsic semantic and patterns of graph data. However, we discover an interesting phenomenon: GCLs can achieve competitive performance even without positive samples. This motivates us to revisit the fundamental mechanism of positive samples in GCLs. From the perspective of Dirichlet energy, we theoretically finds that message passing, a key mechanism in graph encoders, trivializes the maximization of positive samples, preventing GCLs from effectively learning from positive samples. To address this, we propose SPGCL to mitigate the trivialization caused by message passing and restore the learning efficacy of positive samples. Specifically, we find that high Dirichlet energy features help positive samples provide effective learning signals while low Dirichlet energy features contribute little to positive learning signal but is useful for positive sampling. Based on this, SPGCL propagates only high Dirichlet energy features and uses low energy features to construct a probability matrix for reliable positive sampling. Extensive experiments demonstrate the effectiveness of SPGCL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。