用图拉普拉斯低频特征预训练图神经网络，缓解过平滑问题。
A Graph Laplacian Eigenvector-based Pre-training Method for Graph Neural Networks
- 通过预测图拉普拉斯的低频特征来实现图结构自监督预训练
- 在分子属性预测任务上优于基线模型，提升下游性能
- 新架构有效缓解深层网络过平滑，捕捉长程依赖关系
近年来，自监督图预训练方法是构建鲁棒图基础模型（GFMs）的关键组成部分。基于结构的预训练方法研究不足，但对依赖图结构的下游应用至关重要。此外，传统消息传递图神经网络在加深时容易因过平滑而难以捕捉全局与区域结构。本文提出拉普拉斯特征向量学习模块（LELM），一种基于预测图拉普拉斯低频特征的新型图神经网络预训练模块。LELM引入新颖架构，克服过平滑问题，使模型能够学习长程依赖关系。实验表明，采用该框架预训练的模型在分子属性预测等下游任务中表现优于基线模型。
原文摘要 · Abstract (English)
The development of self-supervised graph pre-training methods is a crucial ingredient in recent efforts to design robust graph foundation models (GFMs). Structure-based pre-training methods are under-explored yet crucial for downstream applications which rely on underlying graph structure. In addition, pre-training traditional message passing GNNs to capture global and regional structure is often challenging due to the risk of oversmoothing as network depth increases. We address these gaps by proposing the Laplacian Eigenvector Learning Module (LELM), a novel pre-training module for graph neural networks (GNNs) based on predicting the low-frequency eigenvectors of the graph Laplacian. Moreover, LELM introduces a novel architecture that overcomes oversmoothing, allowing the GNN model to learn long-range interdependencies. Empirically, we show that models pre-trained via our framework outperform baseline models on downstream molecular property prediction tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。