通过语义分布对齐,提升跨域图预训练的通用性与效果。
LEDA: Latent Semantic Distribution Alignment for Multi-domain Graph Pre-training
- 设计维度投影与变分语义推理模块,实现多域图特征对齐。
- 在少样本跨域场景下性能超越现有基线与通用模型。
- 适合需要跨域泛化能力的图学习任务研究者使用。
近年来,GPT、DeepSeek等通用大模型的发展推动了图预训练向通用性方向演进,旨在利用图表示学习跨多个领域的丰富且可迁移知识,以提升下游任务表现。然而,现有方法在从通用图数据中学习有效知识方面仍面临挑战,主要源于数据对齐方式过于简单以及训练指导不足。前者表现为对高度异构的图数据采用单一统一处理,导致语义错位;后者则源于将领域内预训练范式直接应用于跨域场景,虽能增强单一数据空间的判别性表示,却难以捕捉多图的有效知识。为此,本文提出一种新型通用图预训练模型LEDA(Latent Semantic Distribution Alignment)。首先引入维度投影单元,自适应地将不同领域特征映射至共享语义空间,实现最小信息损失;进而设计变分语义推理模块,获取共享潜在分布,并以此指导域间投影,使对齐过程遵循跨域共性语义,保障跨域语义学习。实验表明,LEDA在多种图结构和下游任务上均表现出色,在少样本跨域设置下显著优于领域内基线及先进通用预训练模型。
原文摘要 · Abstract (English)
Recent advances in generic large models, such as GPT and DeepSeek, have motivated the introduction of universality to graph pre-training, aiming to learn rich and generalizable knowledge across diverse domains using graph representations to improve performance in various downstream applications. However, most existing methods face challenges in learning effective knowledge from generic graphs, primarily due to simplistic data alignment and limited training guidance. The issue of simplistic data alignment arises from the use of a straightforward unification for highly diverse graph data, which fails to align semantics and misleads pre-training models. The problem with limited training guidance lies in the arbitrary application of in-domain pre-training paradigms to cross-domain scenarios. While it is effective in enhancing discriminative representation in one data space, it struggles to capture effective knowledge from many graphs. To address these challenges, we propose a novel Latent sEmantic Distribution Alignment (LEDA) model for universal graph pre-training. Specifically, we first introduce a dimension projection unit to adaptively align diverse domain features into a shared semantic space with minimal information loss. Furthermore, we design a variational semantic inference module to obtain the shared latent distribution. The distribution is then adopted to guide the domain projection, aligning it with shared semantics across domains and ensuring cross-domain semantic learning. LEDA exhibits strong performance across a broad range of graphs and downstream tasks. Remarkably, in few-shot cross-domain settings, it significantly outperforms in-domain baselines and advanced universal pre-training models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。