用类别信息做伪标签,提升材料性质预测精度。
Supervised Pretraining for Material Property Prediction
- 利用已有类别信息作为伪标签进行有监督预训练。
- 在6个任务上误差降低2%至6.67%,刷新性能基准。
- 适合材料科学与深度学习交叉研究者参考。
准确预测材料性质有助于发现具有特定功能的新材料。深度学习模型近年来在捕捉结构-性质关系方面展现出优异的准确性和灵活性。然而,这些模型通常依赖于监督学习,需要大量标注数据,而这一过程成本高昂且耗时。自监督学习(SSL)通过在大规模无标签数据上预训练,构建可微调的基础模型,提供了一种有前景的替代方案。本文提出有监督预训练方法,即使下游任务涉及无关材料性质,也可利用现有类别信息作为代理标签引导学习。我们在两种先进的SSL模型上评估该策略,并引入一种新的有监督预训练框架。为进一步增强表示学习,我们提出一种基于图的增强技术,在不改变材料图结构的前提下注入噪声以提高鲁棒性。最终得到的基础模型在六个具有挑战性的材料性质预测任务上进行微调,相比基线模型取得显著性能提升,平均绝对误差(MAE)降低2%至6.67%,建立了新材料性质预测的新基准。本研究首次探索了在材料性质预测中使用代理标签的有监督预训练,推动了该领域的方法与应用发展。
原文摘要 · Abstract (English)
Accurate prediction of material properties facilitates the discovery of novel materials with tailored functionalities. Deep learning models have recently shown superior accuracy and flexibility in capturing structure-property relationships. However, these models often rely on supervised learning, which requires large, well-annotated datasets an expensive and time-consuming process. Self-supervised learning (SSL) offers a promising alternative by pretraining on large, unlabeled datasets to develop foundation models that can be fine-tuned for material property prediction. In this work, we propose supervised pretraining, where available class information serves as surrogate labels to guide learning, even when downstream tasks involve unrelated material properties. We evaluate this strategy on two state-of-the-art SSL models and introduce a novel framework for supervised pretraining. To further enhance representation learning, we propose a graph-based augmentation technique that injects noise to improve robustness without structurally deforming material graphs. The resulting foundation models are fine-tuned for six challenging material property predictions, achieving significant performance gains over baselines, ranging from 2% to 6.67% improvement in mean absolute error (MAE) and establishing a new benchmark in material property prediction. This study represents the first exploration of supervised pertaining with surrogate labels in material property prediction, advancing methodology and application in the field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。