arXiv:2504.12011cs.LGcs.AI2025-04中稿 · the Web Conference被引 11

通过信息论分解平衡图嵌入平滑性,提升自监督学习性能

Balancing Graph Embedding Smoothness in Self-Supervised Learning via Information-Theoretic Decomposition

  • 用信息论框架拆分自监督目标,识别平滑性失衡根源
  • 提出BSG框架,三类损失协同优化,显著提升多任务表现
  • 适合关注图神经网络自监督学习与嵌入质量的研究者

图的自监督学习近年来受到广泛关注,尤其在将为其他领域设计的预训练任务(如对比学习、特征重建)应用于图神经网络时。然而,这些方法是否有效捕捉图的核心属性——表示相似性与邻居一致——仍不明确。我们发现现有方法在图嵌入平滑性上呈现两极分化,各自在特定下游任务中占优。通过引入邻域表示变量的信息论框架,将自监督目标分解为三项,揭示这种分化源于各成分间的不平衡,而现有方法未能有效维持。进一步分析表明,适度平衡两端可提升多种下游任务的表现。为此,我们提出BSG(Balancing Smoothness in Graph SSL)框架,设计新型损失函数以协同优化三项:邻域损失、最小化损失和发散损失。理论分析阐明了各项损失在自监督学习与图平滑性上的作用。在多个真实世界数据集上的节点分类与链接预测任务中,实验结果一致表明BSG达到当前最优性能。代码已公开于 https://github.com/steve30572/BSG。

原文摘要 · Abstract (English)

Self-supervised learning (SSL) in graphs has garnered significant attention, particularly in employing Graph Neural Networks (GNNs) with pretext tasks initially designed for other domains, such as contrastive learning and feature reconstruction. However, it remains uncertain whether these methods effectively reflect essential graph properties, precisely representation similarity with its neighbors. We observe that existing methods position opposite ends of a spectrum driven by the graph embedding smoothness, with each end corresponding to outperformance on specific downstream tasks. Decomposing the SSL objective into three terms via an information-theoretic framework with a neighbor representation variable reveals that this polarization stems from an imbalance among the terms, which existing methods may not effectively maintain. Further insights suggest that balancing between the extremes can lead to improved performance across a wider range of downstream tasks. A framework, BSG (Balancing Smoothness in Graph SSL), introduces novel loss functions designed to supplement the representation quality in graph-based SSL by balancing the derived three terms: neighbor loss, minimal loss, and divergence loss. We present a theoretical analysis of the effects of these loss functions, highlighting their significance from both the SSL and graph smoothness perspectives. Extensive experiments on multiple real-world datasets across node classification and link prediction consistently demonstrate that BSG achieves state-of-the-art performance, outperforming existing methods. Our implementation code is available at https://github.com/steve30572/BSG.

图神经网络自监督学习嵌入平滑性信息论

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。