arXiv:2506.22510cs.CLcs.AI2025-06被引 4

提出新框架,让图模型更好跨领域学习,不依赖文本也能提升性能。

Towards Text-free Graph Foundation Models: Rethinking Multi-Domain Graph Contrastive Learning

  • 通过域差异识别与域标记编码,改进多领域图对比学习。
  • 在5个数据集上最高提升19.33%准确率,19.13%宏F1分数。
  • 适合做无文本图数据的跨领域迁移任务的研究者使用。

基础模型在自然语言处理和计算机视觉中取得巨大成功,其核心在于预训练阶段融合多领域知识并迁移到目标领域。针对现实应用中大量存在无文本特征的图数据(如社交网络、推荐系统),研究者尝试将该范式扩展至图领域,构建图基础模型。然而,与CV和NLP不同,不同领域的图在语义和属性上差异巨大,而现有方法仍沿用单领域设计的传统对比学习策略,将跨域样本视为等价,导致难以有效吸收各领域知识。实验发现,固有的域间差异严重阻碍了表示学习。为此,本文提出新型多域预训练与跨域迁移框架MDGCL:在预训练阶段,设计能识别并捕捉域差异的对比学习策略,并引入域标记编码全局域信息;在下游阶段,采用域注意力机制实现细粒度域知识迁移。在五个基准数据集上的大量实验表明,本方法显著优于当前最先进方法,准确率最高提升19.33%,宏F1分数最高提升19.13%。

原文摘要 · Abstract (English)

Foundation models have achieved great success in natural language processing (NLP) and computer vision (CV). Their success largely stems from the ability to integrate multi-domain knowledge in pre-training and transfer it to target domains. Considering graph data, especially graphs without textual features, is ubiquitous in real-world applications such as social networks and recommendation systems, some researchers have attempted to extend this paradigm to the graph field, aiming to construct graph foundation models. However, unlike CV and NLP, there are huge gaps among the semantics and properties of graphs in different domains, while current works still adopt traditional contrastive pre-training strategies designed in the single-domain scenario, which regard contrastive samples from different domains as equivalent. From experimental investigations, we discovered that inherent domain-specific differences prevent these strategies from effectively absorbing knowledge from different domains to generate informative representations. In this paper, we propose a novel multi-domain pre-training and cross-domain transfer framework, namely MDGCL.In the pre-training stage, we design a contrastive learning strategy to substantially recognize and capture domain differences, and introduce domain tokens to encode domain-level global information. In the downstream stage, we introduce a domain attention mechanism to enable fine-grained domain knowledge transfer. Extensive experiments on five benchmark datasets have demonstrated that our method outperforms state-of-the-art significantly, with the maximum improvement of 19.33\% on accuracy and 19.13\% on Macro-F1 score.

图神经网络多领域学习对比学习跨域迁移

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。