arXiv:2504.10700cs.DCcs.AI2025-04中稿 · The 34th ACM Inter…被引 5

优化分子模型训练的数据分发与计算内核,提速近6倍。

Optimizing Data Distribution and Kernel Performance for Efficient Training of Chemistry Foundation Models: A Case Study with MACE

  • 将数据分发建模为多目标装箱问题,迭代求解实现负载均衡。
  • 定位并优化对称张量收缩计算内核,显著提升训练效率。
  • 适合大规模分子基础模型训练的高性能计算研究者参考。

基于图神经网络(GNN)的化学基础模型(CFMs)在计算化学和材料科学中日益重要,通过处理三维分子图结构来理解物质并发现新材料。与处理大规模同质图的GNN不同,CFMs需处理大量尺寸各异的几何图,需采用不同的优化策略。本文针对MACE——一种前沿化学基础模型——在训练中的两个关键阶段:数据分发与模型训练,提出优化方案。我们通过将数据分发问题建模为多目标装箱问题,设计了一种快速、高效的迭代算法,实现良好的负载均衡。在训练阶段,识别出对称张量收缩是MACE的核心计算内核,并对其进行了优化。结合数据分发均衡与内核优化,显著提升了训练效率。实验表明,在740个GPU上使用260万样本数据集时,单轮训练时间从12分钟降至2分钟,实现近6倍加速。

原文摘要 · Abstract (English)

Chemistry Foundation Models (CFMs) that leverage Graph Neural Networks (GNNs) operating on 3D molecular graph structures are becoming indispensable tools for computational chemists and materials scientists. These models facilitate the understanding of matter and the discovery of new molecules and materials. In contrast to GNNs operating on a large homogeneous graphs, GNNs used by CFMs process a large number of geometric graphs of varying sizes, requiring different optimization strategies than those developed for large homogeneous GNNs. This paper presents optimizations for two critical phases of CFM training: data distribution and model training, targeting MACE - a state-of-the-art CFM. We address the challenge of load balancing in data distribution by formulating it as a multi-objective bin packing problem. We propose an iterative algorithm that provides a highly effective, fast, and practical solution, ensuring efficient data distribution. For the training phase, we identify symmetric tensor contraction as the key computational kernel in MACE and optimize this kernel to improve the overall performance. Our combined approach of balanced data distribution and kernel optimization significantly enhances the training process of MACE. Experimental results demonstrate a substantial speedup, reducing per-epoch execution time for training from 12 to 2 minutes on 740 GPUs with a 2.6M sample dataset.

分子建模图神经网络性能优化高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。