arXiv:2509.21811cs.LG2025-09

发现神经材料模型的性能增长规律,指导如何高效提升预测精度。

Scaling Laws for Neural Material Models

  • 通过实验确定数据、模型规模和算力三者的性能增长规律。
  • 损失值随训练数据量按幂律下降,幂律指数β可量化模型收益。
  • 适用于电池、半导体等材料设计,为科研与工业提供可扩展方案。

预测材料属性对改进电池、半导体和医疗设备至关重要。深度学习通过快速预测能量、力和应力,帮助科学家发现有潜力的材料。企业已在语言建模等领域大规模投入深度学习模型。本文研究神经网络在材料属性预测中,训练数据量、模型规模和计算资源三者对性能的影响。我们训练了Transformer与EquiformerV2模型,发现其性能遵循经验幂律:损失值 $L = α hinspace N^{-β}$,其中 $N$ 为任一超参数,$α$ 与 $β$ 为常数。同时支持通过命令行调整训练设置,如训练轮数、最大学习率和混合精度开关。未来可拓展至GemNet及全连接网络等模型,比较其缩放特性。

原文摘要 · Abstract (English)

Predicting material properties is crucial for designing better batteries, semiconductors, and medical devices. Deep learning helps scientists quickly find promising materials by predicting their energy, forces, and stresses. Companies scale capacities of deep learning models in multiple domains, such as language modeling, and invest many millions of dollars into such models. Our team analyzes how scaling training data (giving models more information to learn from), model sizes (giving models more capacity to learn patterns), and compute (giving models more computational resources) for neural networks affects their performance for material property prediction. In particular, we trained both transformer and EquiformerV2 neural networks to predict material properties. We find empirical scaling laws for these models: we can predict how increasing each of the three hyperparameters (training data, model size, and compute) affects predictive performance. In particular, the loss $L$ can be measured with a power law relationship $L = α\cdot N^{-β}$, where $α$ and $β$ are constants while $N$ is the relevant hyperparameter. We also incorporate command-line arguments for changing training settings such as the amount of epochs, maximum learning rate, and whether mixed precision is enabled. Future work could entail further investigating scaling laws for other neural network models in this domain, such as GemNet and fully connected networks, to assess how they compare to the models we trained.

材料预测神经网络缩放定律

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。