用黎曼几何统一多领域图数据,提升图模型的知识迁移能力。
Multi-Domain Riemannian Graph Gluing for Building Graph Foundation Models
- 通过自适应正交基表征局部图结构,再将其粘合成平滑流形。
- 在多个图数据集上实现更优的预训练效果,转移性能显著提升。
- 适合研究图神经网络、知识迁移与基础模型构建的学者。
多领域图预训练通过融合不同领域的知识来提升目标领域的性能,对构建图基础模型至关重要。尽管已有进展,但现有方法难以回答一个根本问题:知识如何跨领域整合或迁移?这一理论局限促使我们重新思考预训练与领域自适应之间的一致性与可迁移性。本文提出一种新的黎曼几何视角,核心思想是将任意图数据集合并为统一且光滑的黎曼流形,从而系统理解知识整合与迁移机制。关键贡献在于建立了神经流形粘合理论:先用自适应正交基刻画局部几何,再将局部块粘合成整体。基于此,我们提出GraphGlue框架,支持批量预训练与EMA原型建模,并提供基于几何一致性的事后可迁移性度量。大量实验表明其在多种图领域均表现优异。此外,我们实证验证了图流形的几何尺度规律:数据量越大,流形越平滑,模型迁移能力越强。代码已开源。
原文摘要 · Abstract (English)
Multi-domain graph pre-training integrates knowledge from diverse domains to enhance performance in the target domains, which is crucial for building graph foundation models. Despite initial success, existing solutions often fall short of answering a fundamental question: how is knowledge integrated or transferred across domains? This theoretical limitation motivates us to rethink the consistency and transferability between model pre-training and domain adaptation. In this paper, we propose a fresh Riemannian geometry perspective, whose core idea is to merge any graph dataset into a unified, smooth Riemannian manifold, enabling a systematic understanding of knowledge integration and transfer. To achieve this, our key contribution is the theoretical establishment of neural manifold gluing, which first characterizes local geometry using an adaptive orthogonal frame and then "glues" the local pieces together into a coherent whole. Building on this theory, we present the GraphGlue framework, which supports batched pre-training with EMA prototyping and provides a transferability measure based on geometric consistence. Extensive experiments demonstrate its superior performance across diverse graph domains. Moreover, we empirically validated GraphGlue's geometric scaling law, showing that larger quantities of datasets improve model transferability by producing a smoother manifold. Codes are available at https://github.com/RiemannGraph/GraphGlue.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。