arXiv:2510.01248cs.CL2025-10NeurIPS被引 5

用文本统一表示图结构,实现跨域图学习的自监督新方法

SSTAG: Structure-Aware Self-Supervised Learning Method for Text-Attributed Graphs

  • 以文本为媒介,融合大模型语义与图网络结构建模能力
  • 双知识蒸馏+内存锚点机制,提升跨域泛化与推理效率
  • 适合资源受限场景下的大规模图数据迁移学习

大规模预训练模型已革新自然语言处理与计算机视觉领域,展现出卓越的跨域泛化能力。然而,在图学习中,模型通常在单一图数据集上训练,限制了知识跨图与任务的迁移能力,且严重依赖大量标注数据,这在资源受限环境下构成挑战。相较于NLP与CV,图结构数据因领域特异性特征空间与结构多样性而面临独特难题。为此,我们提出一种面向文本属性图(Text Attributed Graphs, TAGs)的结构感知自监督学习方法SSTAG。通过将文本作为图学习的统一表征媒介,SSTAG连接大语言模型(LLMs)的语义推理与图神经网络(GNNs)的结构建模能力。该方法引入双重知识蒸馏框架,将LLMs与GNNs共同蒸馏至结构感知多层感知机(MLPs),增强大规模TAGs的可扩展性;同时设计内存机制,将典型图表示存储于内存库中,并与内存锚点对齐,整合不变知识,提升模型泛化能力。大量实验表明,SSTAG在跨域迁移学习任务中优于现有最优模型,具备优异可扩展性,同时降低推理成本并保持竞争力表现。

原文摘要 · Abstract (English)

Large scale pretrained models have revolutionized Natural Language Processing (NLP) and Computer Vision (CV), showcasing remarkable cross domain generalization abilities. However, in graph learning, models are typically trained on individual graph datasets, limiting their capacity to transfer knowledge across different graphs and tasks. This approach also heavily relies on large volumes of annotated data, which presents a significant challenge in resource-constrained settings. Unlike NLP and CV, graph structured data presents unique challenges due to its inherent heterogeneity, including domain specific feature spaces and structural diversity across various applications. To address these challenges, we propose a novel structure aware self supervised learning method for Text Attributed Graphs (SSTAG). By leveraging text as a unified representation medium for graph learning, SSTAG bridges the gap between the semantic reasoning of Large Language Models (LLMs) and the structural modeling capabilities of Graph Neural Networks (GNNs). Our approach introduces a dual knowledge distillation framework that co-distills both LLMs and GNNs into structure-aware multilayer perceptrons (MLPs), enhancing the scalability of large-scale TAGs. Additionally, we introduce an in-memory mechanism that stores typical graph representations, aligning them with memory anchors in an in-memory repository to integrate invariant knowledge, thereby improving the model's generalization ability. Extensive experiments demonstrate that SSTAG outperforms state-of-the-art models on cross-domain transfer learning tasks, achieves exceptional scalability, and reduces inference costs while maintaining competitive performance.

图神经网络自监督学习文本属性图知识蒸馏

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。