统一预训练异构与同构图,提升跨类型迁移性能。
Unified Multi-Domain Graph Pre-training for Homogeneous and Heterogeneous Graphs via Domain-Specific Expert Encoding
- 设计统一多视角图构建,无需区分图类型即可编码
- 引入领域专属专家,缓解混合图间的分布差异
- 任务导向专家融合策略,自适应提升下游表现
图预训练近年来取得显著进展,为下游任务提供可迁移的表示。然而,现有方法大多仅适用于同构图或异构图,难以实现跨图类型的统一建模。这与真实场景中同构与异构图并存、上下游分布差异普遍的情况相悖。本文实证表明,混合同构与异构图进行预训练有助于下游任务,并提出统一的多域图预训练方法(GPH²)。通过统一多视图图构建,同时编码两类图而无需显式类型设计;针对混合图带来的跨域分布差异,引入领域专属专家,每个专家独立在单一图上预训练以捕捉领域知识,从而保护主编码器免受分布偏移影响。针对下游任务,设计任务导向的专家融合策略,根据专家判别能力自适应集成。在混合图上的大量实验表明,GPH² 能稳定实现跨图类型与跨域迁移,显著优于现有方法。
原文摘要 · Abstract (English)
Graph pre-training has achieved remarkable success in recent years, delivering transferable representations for downstream adaptation. However, most existing methods are designed for either homogeneous or heterogeneous graphs, thereby hindering unified graph modeling across diverse graph types. This separation contradicts real-world applications, where mixed homogeneous and heterogeneous graphs are ubiquitous, and distribution shifts between upstream pre-training and downstream deployment are common. In this paper, we empirically demonstrate that a balanced mixture of homogeneous and heterogeneous graph pre-training benefits downstream tasks and propose a unified multi-domain \textbf{G}raph \textbf{P}re-training method across \textbf{H}omogeneous and \textbf{H}eterogeneous graphs ($\mathbf{GPH^{2}}$). To address the lack of a unified encoder for homogeneous and heterogeneous graphs, we propose a Unified Multi-View Graph Construction that simultaneously encodes both without explicit graph-type-specific designs. To cope with the increased cross-domain distribution discrepancies arising from mixed graphs, we introduce domain-specific expert encoding. Each expert is independently pre-trained on a single graph to capture domain-specific knowledge, thereby shielding the pre-training encoder from the adverse effects of cross-domain discrepancies. For downstream tasks, we further design a Task-oriented Expert Fusion Strategy that adaptively integrates multiple experts based on their discriminative strengths. Extensive experiments on mixed graphs demonstrate that $\text{GPH}^{2}$ enables stable transfer across graph types and domains, significantly outperforming existing graph pre-training methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。