arXiv:2409.11129cs.LGcs.DB2024-09被引 11

重排图结构可显著加速GNN训练,尤其在GPU上效果更佳。

Can Graph Reordering Speed Up Graph Neural Network Training? An Experimental Study

  • 通过12种重排策略优化图数据布局,改善内存访问效率。
  • 在CPU和GPU上均减少训练时间,轻量级重排在GPU上更优。
  • 超参数与重排指标影响效果,重排成本通常可被收益抵消。

图神经网络(GNN)能处理图结构数据,但大规模图的训练因高维邻居特征迭代聚合与稀疏结构导致内存访问效率低、训练耗时长。图重排是一种优化图数据布局的策略,虽在图分析中有效,但对GNN训练的影响尚未明确。本文在PyTorch Geometric与Deep Graph Library两个主流GNN系统中,对12种重排策略进行实证评估。结果表明,图重排能有效缩短CPU与GPU上的训练时间。进一步发现:GNN超参数影响重排效果;重排指标对策略选择至关重要;轻量级重排在GPU上优于CPU;投入的重排时间在多数情况下可被训练加速所抵消。

原文摘要 · Abstract (English)

Graph neural networks (GNNs) are a type of neural network capable of learning on graph-structured data. However, training GNNs on large-scale graphs is challenging due to iterative aggregations of high-dimensional features from neighboring vertices within sparse graph structures combined with neural network operations. The sparsity of graphs frequently results in suboptimal memory access patterns and longer training time. Graph reordering is an optimization strategy aiming to improve the graph data layout. It has shown to be effective to speed up graph analytics workloads, but its effect on the performance of GNN training has not been investigated yet. The generalization of reordering to GNN performance is nontrivial, as multiple aspects must be considered: GNN hyper-parameters such as the number of layers, the number of hidden dimensions, and the feature size used in the GNN model, neural network operations, large intermediate vertex states, and GPU acceleration. In our work, we close this gap by performing an empirical evaluation of 12 reordering strategies in two state-of-the-art GNN systems, PyTorch Geometric and Deep Graph Library. Our results show that graph reordering is effective in reducing training time for CPU- and GPU-based training, respectively. Further, we find that GNN hyper-parameters influence the effectiveness of reordering, that reordering metrics play an important role in selecting a reordering strategy, that lightweight reordering performs better for GPU-based than for CPU-based training, and that invested reordering time can in many cases be amortized.

图神经网络性能优化图重排GPU加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。