跨域图模型融合，提升模型泛化能力
Out-of-Distribution Graph Models Merging
- 通过生成混合分布实现多领域图模型融合
- 无需源/目标域数据，支持任意架构
- 适用于不同图神经网络结构的通用合并
本文研究了一种新的跨分布图模型融合问题，旨在从多个在不同领域预训练、存在分布差异的图模型中构建一个具有泛化能力的统一模型。该问题挑战在于难以隐式学习模型参数中的领域不变知识，并整合可能异构的图神经网络主干网络的专业能力。为此，我们提出一种图生成策略，用于实例化多个领域的混合分布。随后，利用MoE模块和掩码机制对预训练图模型进行合并与微调，实现广义适应。所提框架具备架构无关性，可在无需任何源或目标域数据的情况下运行。理论分析与实验结果均证明该方法在解决模型泛化问题上的有效性。
原文摘要 · Abstract (English)
This paper studies a novel problem of out-of-distribution graph models merging, which aims to construct a generalized model from multiple graph models pre-trained on different domains with distribution discrepancy. This problem is challenging because of the difficulty in learning domain-invariant knowledge implicitly in model parameters and consolidating expertise from potentially heterogeneous GNN backbones. In this work, we propose a graph generation strategy that instantiates the mixture distribution of multiple domains. Then, we merge and fine-tune the pre-trained graph models via a MoE module and a masking mechanism for generalized adaptation. Our framework is architecture-agnostic and can operate without any source/target domain data. Both theoretical analysis and experimental results demonstrate the effectiveness of our approach in addressing the model generalization problem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。