提出首个跨域图结构编码器,可通用捕捉不同图的拓扑特征。
Towards A Universal Graph Structural Encoder
- 基于图Transformer设计,用结构信息引导注意力机制
- 在多类图数据上预训练,显著提升下游任务性能
- 适合需少调参的通用图表示学习场景
大规模预训练在学习可迁移表示方面展现出巨大潜力。然而,在图领域中,如何在不同图域间捕捉并传递结构信息仍具挑战,主要源于各类图拓扑模式的根本差异——例如社交网络与商品共购图结构迥异。此外,现有模型难以充分捕捉图结构的复杂性,导致图嵌入空间探索不足。为此,我们提出GFSE,首个基于多自监督学习目标的跨域图结构编码器,适用于网页图、社交网络和引用网络等多样化场景。基于图Transformer构建,GFSE引入由图结构信息指导的注意力机制,能够编码复杂图中多层次、细粒度的拓扑特征。预训练后的GFSE生成通用且理论表达性强的位置与结构编码,可无缝集成至多种下游图特征编码器,包括用于向量特征的图神经网络及用于文本属性图的大语言模型(LLMs)。在合成与真实世界数据集上的大量实验表明,GFSE能显著提升模型性能,同时大幅减少任务特定微调需求。
原文摘要 · Abstract (English)
Recent advancements in large-scale pre-training have shown the potential to learn generalizable representations for downstream tasks. In the graph domain, however, capturing and transferring structural information across different graph domains remains challenging, primarily due to the inherent differences in graph topological patterns across various contexts. For example, a social network's structure is fundamentally different from that of a product co-purchase graph. Additionally, most existing models struggle to capture the rich topological complexity of graph structures, leading to inadequate exploration of the graph embedding space. To address these challenges, we propose GFSE, a universal pre-trained graph encoder designed to capture transferable structural patterns across diverse domains such as the web graph, social networks, and citation networks. GFSE is the first cross-domain graph structural encoder pre-trained with multiple self-supervised learning objectives. Built on a Graph Transformer, GFSE incorporates attention mechanisms informed by graph structural information, enabling it to encode intricate multi-level and fine-grained topological features within complex graph structures. The pre-trained GFSE produces generic and theoretically expressive positional and structural encoding for graphs, which can be seamlessly integrated with various downstream graph feature encoders, including graph neural networks for vectorized features and Large Language Models (LLMs) for text-attributed graphs. Comprehensive experiments on synthetic and real-world datasets demonstrate GFSE's capability to significantly enhance the model's performance while requiring substantially less task-specific fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。