arXiv:2504.08112cs.LGcond-mat.mtrl-sci2025-04中稿 · DAC'25被引 6

探索原子材料建模中图神经网络的规模化规律,构建百亿参数大模型。

Scaling Laws of Graph Neural Networks for Atomistic Materials Modeling

  • 基于大语言模型技术构建超大规模图神经网络,实现高效训练与部署。
  • 发现模型规模、数据量与预测精度之间存在可量化的增长规律。
  • 为材料科学领域提供可扩展的高性能图神经网络框架,适合研究者复用。

原子材料建模在药物发现与材料科学等领域具有重要意义,准确预测材料性质可推动科学突破。图神经网络(GNN)因其捕捉复杂关系结构的能力,已成为该领域的前沿方法。尽管机器学习性能随模型和数据规模增长而提升,但当前用于原子材料建模的GNN仍远小于大语言模型(LLMs),后者使用数十亿参数和千兆字节级数据实现卓越表现。为此,本文通过构建具有百亿参数、基于千兆字节级数据的奠基性模型,探索了原子材料建模中GNN的规模化极限。研究引入来自LLM库的技术,实现大规模数据与模型的有效管理,从而支持高效训练与部署。本工作回答了三个核心问题:模型架构的可扩展性、数据规模对精度的影响,以及LLM技术在GNN中的适用性。结果包括:(1) 揭示了模型大小、数据量与准确率之间的量化增长规律;(2) 提出一个专为原子材料建模优化的奠基性GNN模型;(3) 开发一个集成先进LLM训练技术的GNN代码库。研究为未来百亿参数、千兆字节级数据的可扩展图神经网络奠定基础。

原文摘要 · Abstract (English)

Atomistic materials modeling is a critical task with wide-ranging applications, from drug discovery to materials science, where accurate predictions of the target material property can lead to significant advancements in scientific discovery. Graph Neural Networks (GNNs) represent the state-of-the-art approach for modeling atomistic material data thanks to their capacity to capture complex relational structures. While machine learning performance has historically improved with larger models and datasets, GNNs for atomistic materials modeling remain relatively small compared to large language models (LLMs), which leverage billions of parameters and terabyte-scale datasets to achieve remarkable performance in their respective domains. To address this gap, we explore the scaling limits of GNNs for atomistic materials modeling by developing a foundational model with billions of parameters, trained on extensive datasets in terabyte-scale. Our approach incorporates techniques from LLM libraries to efficiently manage large-scale data and models, enabling both effective training and deployment of these large-scale GNN models. This work addresses three fundamental questions in scaling GNNs: the potential for scaling GNN model architectures, the effect of dataset size on model accuracy, and the applicability of LLM-inspired techniques to GNN architectures. Specifically, the outcomes of this study include (1) insights into the scaling laws for GNNs, highlighting the relationship between model size, dataset volume, and accuracy, (2) a foundational GNN model optimized for atomistic materials modeling, and (3) a GNN codebase enhanced with advanced LLM-based training techniques. Our findings lay the groundwork for large-scale GNNs with billions of parameters and terabyte-scale datasets, establishing a scalable pathway for future advancements in atomistic materials modeling.

图神经网络材料建模规模化大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。