用梯度优化法加速图神经网络融合，提升性能并大幅降低内存占用。
Enhanced Soups for Graph Neural Networks
- 基于梯度下降的新型融合策略，显著减少计算与内存开销。
- 在多个OGB数据集上实现最高1.2%精度提升和2.1倍加速。
- 分块版本可减少76%内存使用，ogbn-products上提速24.5倍。
图神经网络（GNN）在众多科学与高性能计算（HPC）应用中表现优异。近期研究表明，将独立训练的GNN模型融合为单一模型可在不增加推理时计算与内存成本的前提下提升性能。然而，现有融合算法通常速度慢且内存消耗大，限制了其可扩展性。本文提出一种基于梯度下降的图神经网络学习型融合方法（Learned Souping），相比现有方法显著降低了时间和内存开销。该方法在多个Open Graph Benchmark（OGB）数据集及GNN架构上进行了评估，实现了最高1.2%的精度提升和2.1倍的加速。此外，我们提出一种新型分块式学习型融合（Partition Learned Souping），显著降低内存使用。在ogbn-products数据集上，结合GraphSAGE模型时，该方法实现了24.5倍加速和76%的内存减少，且未牺牲精度。
原文摘要 · Abstract (English)
Graph Neural Networks (GNN) have demonstrated state-of-the-art performance in numerous scientific and high-performance computing (HPC) applications. Recent work suggests that "souping" (combining) individually trained GNNs into a single model can improve performance without increasing compute and memory costs during inference. However, existing souping algorithms are often slow and memory-intensive, which limits their scalability. We introduce Learned Souping for GNNs, a gradient-descent-based souping strategy that substantially reduces time and memory overhead compared to existing methods. Our approach is evaluated across multiple Open Graph Benchmark (OGB) datasets and GNN architectures, achieving up to 1.2% accuracy improvement and 2.1X speedup. Additionally, we propose Partition Learned Souping, a novel partition-based variant of learned souping that significantly reduces memory usage. On the ogbn-products dataset with GraphSAGE, partition learned souping achieves a 24.5X speedup and a 76% memory reduction without compromising accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。