对比静态与动态批处理，发现可实现2.7倍训练加速
Training speedups via batching for geometric learning: an analysis of static and dynamic algorithms
- 比较静态与动态批处理策略在图模型中的表现
- 实验显示最快算法取决于数据、模型和硬件组合
- 适合优化图神经网络训练效率的研究者参考
图神经网络（GNN）在材料科学、化学和社科等领域展现出良好效果。由于参数量常达数百万，类似其他神经网络，通常以小批量方式加载图数据进行参数更新。尽管批处理对普通神经网络的训练时间和性能影响已深入研究，但对GNN的影响仍不明确。本文分析了两种图模型的批处理算法：静态与动态批处理，针对小型分子数据集QM9和AFLOW材料数据库展开实验。结果表明，改变批处理策略最高可带来2.7倍的训练加速；然而最优算法取决于数据、模型、批大小、硬件及训练步数等多重因素。部分特定组合下，静态与动态批处理在模型学习指标上表现出显著差异。
原文摘要 · Abstract (English)
Graph neural networks (GNN) have shown promising results for several domains such as materials science, chemistry, and the social sciences. GNN models often contain millions of parameters, and like other neural network (NN) models, are often fed only a fraction of the graphs that make up the training dataset in batches to update model parameters. The effect of batching algorithms on training time and model performance has been thoroughly explored for NNs but not yet for GNNs. We analyze two different batching algorithms for graph-based models, namely static and dynamic batching for two datasets, the QM9 dataset of small molecules and the AFLOW materials database. Our experiments show that changing the batching algorithm can provide up to a 2.7x speedup, but the fastest algorithm depends on the data, model, batch size, hardware, and number of training steps run. Experiments show that for a select number of combinations of batch size, dataset, and model, significant differences in model learning metrics are observed between static and dynamic batching algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。