用材料文献训练大模型,让AI能帮科学家发现新晶体。
Foundational Large Language Models for Materials Research
- 在材料文献和晶格数据上继续预训练,打造专用大模型LLaMat。
- LLaMat-CIF可高效生成稳定晶体,覆盖全元素周期表。
- 小模型反而表现更好,提示过度训练可能影响适应性。
材料发现对应对全球挑战至关重要,但材料科学文献的爆炸式增长带来了知识提取与科学推理的瓶颈。大语言模型(LLMs)为自动化分析与预测提供了新机遇,但需领域适配才能有效应用。本文提出基于LLaMA模型在大规模材料文献与晶格数据上持续预训练的LLaMat系列模型。系统评估表明,LLaMat在材料文本处理与结构化信息提取上表现优异,同时保持通用语言能力。其中,专用于晶格文件(CIF)的LLaMat-CIF展现出前所未有的晶体结构生成能力,可高覆盖率预测周期表中各类元素的稳定晶体。尽管LLaMA-3性能更优,但LLaMat-2在多项材料任务中表现出意料之外的更强领域适应性,包括文本与表格信息抽取、晶体生成等,暗示过度训练可能导致模型适应性下降。本研究证明了领域适配对构建可部署的材料研究智能协作者的有效性,并为科学领域专用AI系统的开发提供重要启示。
原文摘要 · Abstract (English)
Materials discovery and development are critical for addressing global challenges. Yet, the exponential growth in materials science literature comprising vast amounts of textual data has created significant bottlenecks in knowledge extraction, synthesis, and scientific reasoning. Large Language Models (LLMs) offer unprecedented opportunities to accelerate materials research through automated analysis and prediction. Still, their effective deployment requires domain-specific adaptation for understanding and solving domain-relevant tasks. Here, we present LLaMat, a family of foundational models for materials science developed through continued pretraining of LLaMA models on an extensive corpus of materials literature and crystallographic data. Through systematic evaluation, we demonstrate that LLaMat excels in materials-specific NLP and structured information extraction while maintaining general linguistic capabilities. The specialized LLaMat-CIF variant demonstrates unprecedented capabilities in crystal structure generation, predicting stable crystals with high coverage across the periodic table. Intriguingly, despite LLaMA-3's superior performance in comparison to LLaMA-2, we observe that LLaMat-2 demonstrates unexpectedly enhanced domain-specific performance across diverse materials science tasks, including structured information extraction from text and tables, more particularly in crystal structure generation, a potential adaptation rigidity in overtrained LLMs. Altogether, the present work demonstrates the effectiveness of domain adaptation towards developing practically deployable LLM copilots for materials research. Beyond materials science, our findings reveal important considerations for domain adaptation of LLMs, such as model selection, training methodology, and domain-specific performance, which may influence the development of specialized scientific AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。