用潜在扩散模型预训练,提升晶体性质预测精度与数据稀缺下的表现
Latent Diffusion Pretraining for Crystal Property Prediction

- 先用变分自编码器将晶体结构映射到平滑潜在空间,再在该空间进行扩散预训练
- 在JARVIS和MP数据集上分别提升4.26%和4.90%,优于从头训练与现有预训练方法
- 适合缺乏标注数据的材料设计场景,尤其对修正DFT误差有帮助
快速准确预测晶体性质是新材料设计的核心挑战。图神经网络和基于Transformer的模型因其能编码原子局部结构环境而成为有力工具,但这些模型依赖大量标注数据,而实际中晶体性质的标签数据稀少。基于扩散模型的预训练-微调策略展现出解决此问题的潜力。本文提出一种新型潜空间扩散预训练框架CrysLDNet,其在预训练阶段结合变分自编码器(VAE)与扩散模型:VAE编码器将3D晶体结构映射至平滑潜在空间,随后在该空间执行扩散过程。该潜空间扩散预训练使图编码器能从大规模无标签数据中有效捕捉结构与化学语义,进而微调用于特定性质预测任务。在多个主流密度泛函理论(DFT)数据集上的实验表明,CrysLDNet显著优于从零训练及已有预训练基线,在JARVIS和MP数据集上分别提升4.26%和4.90%。此外,所学表征在数据稀疏条件下仍具鲁棒性,且微调时使用少量实验数据即可有效修正DFT误差。代码已开源:https://github.com/shrimonmuke0202/CrysLDNet.git。
原文摘要 · Abstract (English)
Fast and accurate prediction of crystal properties is a central challenge in new materials design. Graph neural networks and Transformer-based models have emerged as powerful tools for this task due to their ability to encode the local structural environment of atoms within a crystal. However, these models are data-hungry, and in practice, labeled data for crystal properties are scarce. Pretraining-finetuning strategies, particularly those based on diffusion models, have shown promise in addressing these limitations. In this work, we introduce a novel latent diffusion based pretraining framework, CrysLDNet, designed to mitigate data scarcity. Our approach integrates a Variational Autoencoder (VAE) with a diffusion model during the pretraining stage. The VAE encoder maps 3D crystal structures into a smooth latent space within which the diffusion process is applied. This latent diffusion pretraining enables the graph encoder to effectively capture structural and chemical semantics from large-scale unlabeled data, which can then be finetuned for specific property prediction tasks. Comprehensive experiments on popular DFT datasets for property prediction reveal that CrysLDNet significantly outperforms both training-from-scratch and pretrained baselines, with improvements of 4.26% and 4.90% on the JARVIS and MP datasets, respectively. Additionally, the learned representations remain robust in sparse-data conditions and are expressive enough to correct DFT errors when finetuned with limited experimental data. Code is available at: https://github.com/shrimonmuke0202/CrysLDNet.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。