无需训练数据即可合并GNN模型，提升准确率与效率
GNNMerge: Merging of GNN Models Without Accessing Training Data
- 通过无任务依赖的节点嵌入对齐策略合并GNN模型
- 在多个数据集上比现有方法最高提升24%准确率
- 支持主流GNN架构的解析解，速度比从头训练快100倍以上
模型合并在机器学习中日益重要，可在不访问原始训练数据的情况下将多个训练好的模型整合为单一模型。尽管该方法在计算机视觉和自然语言处理领域已取得成功，但在图神经网络（GNN）中的应用仍属空白。现有方法通常依赖共享初始化假设，而这一假设在GNN中很少成立。本文首次对GNN模型合并算法进行基准测试，揭示其在此领域的表现有限。为此，我们提出GNNMerge，采用一种与任务无关的节点嵌入对齐策略来合并GNN模型。此外，我们证明在温和放宽条件下，所提出的优化目标对于广泛使用的GNN架构可直接求得解析解，显著提升计算效率。在多种数据集、任务和架构上的实证评估表明，GNNMerge相比现有方法最高提升24%准确率，同时比从头训练快超过两个数量级。
原文摘要 · Abstract (English)
Model merging has gained prominence in machine learning as a method to integrate multiple trained models into a single model without accessing the original training data. While existing approaches have demonstrated success in domains such as computer vision and NLP, their application to Graph Neural Networks (GNNs) remains unexplored. These methods often rely on the assumption of shared initialization, which is seldom applicable to GNNs. In this work, we undertake the first benchmarking study of model merging algorithms for GNNs, revealing their limited effectiveness in this context. To address these challenges, we propose GNNMerge, which utilizes a task-agnostic node embedding alignment strategy to merge GNNs. Furthermore, we establish that under a mild relaxation, the proposed optimization objective admits direct analytical solutions for widely used GNN architectures, significantly enhancing its computational efficiency. Empirical evaluations across diverse datasets, tasks, and architectures establish GNNMerge to be up to 24% more accurate than existing methods while delivering over 2 orders of magnitude speed-up compared to training from scratch.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。