预训练图神经网络跨数据集迁移能力有限，需足够下游数据才有效。
Towards Foundation Models on Graphs: An Analysis on Cross-Dataset Transfer of Pretrained GNNs
- 基于结构预训练并融合特征信息，保持对特征的无关性。
- 下游数据量不足时，预训练效果不明显，依赖预训练数据量与性质。
- 特征相似性是提升性能的关键，适合图学习研究者参考。
为探索图基础模型的初步可行性，我们研究了预训练图神经网络在不同数据集间的迁移能力，要求方法对数据集特有特征及其编码保持无偏。基于纯结构预训练方法，我们提出一种扩展以捕捉特征信息，同时保持特征无关性。在不同训练样本量和预训练数据集选择下评估预训练模型在下游任务的表现。初步结果显示，仅当下游数据点足够多时，预训练嵌入才能提升泛化能力，且提升程度取决于预训练数据的数量与特性。特征信息可带来改进，但目前需要预训练与下游特征空间具备一定相似性。
原文摘要 · Abstract (English)
To develop a preliminary understanding towards Graph Foundation Models, we study the extent to which pretrained Graph Neural Networks can be applied across datasets, an effort requiring to be agnostic to dataset-specific features and their encodings. We build upon a purely structural pretraining approach and propose an extension to capture feature information while still being feature-agnostic. We evaluate pretrained models on downstream tasks for varying amounts of training samples and choices of pretraining datasets. Our preliminary results indicate that embeddings from pretrained models improve generalization only with enough downstream data points and in a degree which depends on the quantity and properties of pretraining data. Feature information can lead to improvements, but currently requires some similarities between pretraining and downstream feature spaces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。