无需负样本即可实现图表示均匀分布,提升训练效率
Negative-Free Self-Supervised Gaussian Embedding of Graphs
- 用高斯分布逼近代替负样本,实现无负样本的均匀性约束
- 在7个图基准上表现相当,参数更少、训练更快、内存更低
- 适合追求高效自监督图学习的科研与工程人员
图对比学习(GCL)是近年来兴起的一种无需标签的图自监督学习框架,用于学习具有区分性的节点表示。其核心目标函数依赖于两个关键属性:对齐性(alignment)和均匀性(uniformity),前者使正样本对的表示相近,后者则要求所有表示在超球面上均匀分布,以防止表示坍缩。现有方法通过拉远不同节点的增强视图(负样本对)来实现均匀性,导致计算开销大、内存占用高,并可能引发类别碰撞问题。本文提出一种无需负样本的均匀性优化方法,受启发于:服从归一化各向同性高斯分布的点在单位超球面上均匀分布。因此,可通过最小化学习到的表示分布与各向同性高斯分布之间的距离,来促进表示的均匀性。该方法无需参数化互信息估计器、额外投影层、非对称结构,也无需负样本。在七个图基准上的大量实验表明,本方法在性能相当的前提下,参数更少、训练时间更短、内存消耗更低。
原文摘要 · Abstract (English)
Graph Contrastive Learning (GCL) has recently emerged as a promising graph self-supervised learning framework for learning discriminative node representations without labels. The widely adopted objective function of GCL benefits from two key properties: \emph{alignment} and \emph{uniformity}, which align representations of positive node pairs while uniformly distributing all representations on the hypersphere. The uniformity property plays a critical role in preventing representation collapse and is achieved by pushing apart augmented views of different nodes (negative pairs). As such, existing GCL methods inherently rely on increasing the quantity and quality of negative samples, resulting in heavy computational demands, memory overhead, and potential class collision issues. In this study, we propose a negative-free objective to achieve uniformity, inspired by the fact that points distributed according to a normalized isotropic Gaussian are uniformly spread across the unit hypersphere. Therefore, we can minimize the distance between the distribution of learned representations and the isotropic Gaussian distribution to promote the uniformity of node representations. Our method also distinguishes itself from other approaches by eliminating the need for a parameterized mutual information estimator, an additional projector, asymmetric structures, and, crucially, negative samples. Extensive experiments over seven graph benchmarks demonstrate that our proposal achieves competitive performance with fewer parameters, shorter training times, and lower memory consumption compared to existing GCL methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。