用物理化学知识增强分子语言模型，提升属性预测与生成能力
MolMetaLM: a Physicochemical Knowledge-Guided Molecular Meta Language Model
- 设计分子专用的<主语, 谓语, 宾语>三元组框架，融合物化知识
- 在数千个预训练任务上恢复噪声，实现多任务性能领先
- 适合分子性质预测、生成和优化的研究者使用
现有分子语言模型多沿用自然语言处理中的掩码语言建模或图文生成方法，但分子不仅由原子/键符号构成，更蕴含重要物理化学特性。普通语言模型引入的语法规则对理解分子无益。本文提出新型物化知识引导的分子元语言框架MolMetaLM。设计分子专用的元语言范式，以多个共享主语（分子）的<S,P,O>知识三元组形式表达，强化物化知识与分子间的语义关联。通过引入不同分子知识与噪声，生成数万项预训练任务。通过恢复标记/序列/顺序级噪声，MolMetaLM在大规模基准测试中表现出色，涵盖性质预测、分子生成、构象推断与分子优化任务。本工作为语言模型设计提供了新视角。
原文摘要 · Abstract (English)
Most current molecular language models transfer the masked language model or image-text generation model from natural language processing to molecular field. However, molecules are not solely characterized by atom/bond symbols; they encapsulate important physical/chemical properties. Moreover, normal language models bring grammar rules that are irrelevant for understanding molecules. In this study, we propose a novel physicochemical knowledge-guided molecular meta language framework MolMetaLM. We design a molecule-specialized meta language paradigm, formatted as multiple <S,P,O> (subject, predicate, object) knowledge triples sharing the same S (i.e., molecule) to enhance learning the semantic relationships between physicochemical knowledge and molecules. By introducing different molecular knowledge and noises, the meta language paradigm generates tens of thousands of pretraining tasks. By recovering the token/sequence/order-level noises, MolMetaLM exhibits proficiency in large-scale benchmark evaluations involving property prediction, molecule generation, conformation inference, and molecular optimization. Through MolMetaLM, we offer a new insight for designing language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。