arXiv:2412.07407cs.LG2024-12被引 7

探究图神经网络位置与结构编码的通用性,助力构建图领域基础模型。

Towards Graph Foundation Models: A Study on the Generalization of Positional and Structural Encodings

  • 设计可学习的位置与结构编码,提升图模型泛化能力。
  • 在多数据集上验证编码增强下游任务性能,尤其小样本下效果显著。
  • 为未来图基础模型提供关键组件,适合图学习研究者参考。

近期将位置与结构编码(PSEs)融入图神经网络(GNNs)的研究显著提升了其在各类图学习任务中的表现。然而,这些编码的普适性及其作为图领域基础表示的潜力仍不明确。本文系统评估了可学习PSEs在微调效率、样本规模扩展性及跨数据集泛化能力方面的表现,重点考察其作为通用预训练模型的可行性——即在少量数据和极少微调下快速适应新任务的潜力。同时,分析了所学表征的表达能力,尤其是在增强下游GNN时的效果。大量基准测试与实证分析表明,PSEs普遍能提升下游模型性能,但部分数据集需特定编码增强以达最优。研究揭示了PSEs在构建未来图基础模型中的巨大潜力,并为图学习领域的基础模型讨论提供了新洞见。

原文摘要 · Abstract (English)

Recent advances in integrating positional and structural encodings (PSEs) into graph neural networks (GNNs) have significantly enhanced their performance across various graph learning tasks. However, the general applicability of these encodings and their potential to serve as foundational representations for graphs remain uncertain. This paper investigates the fine-tuning efficiency, scalability with sample size, and generalization capability of learnable PSEs across diverse graph datasets. Specifically, we evaluate their potential as universal pre-trained models that can be easily adapted to new tasks with minimal fine-tuning and limited data. Furthermore, we assess the expressivity of the learned representations, particularly, when used to augment downstream GNNs. We demonstrate through extensive benchmarking and empirical analysis that PSEs generally enhance downstream models. However, some datasets may require specific PSE-augmentations to achieve optimal performance. Nevertheless, our findings highlight their significant potential to become integral components of future graph foundation models. We provide new insights into the strengths and limitations of PSEs, contributing to the broader discourse on foundation models in graph learning.

图神经网络基础模型编码设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。