通过片段编辑训练分子SMILES模型,提升结构理解能力
SMI-Editor: Edit-based SMILES Language Model with Fragment-level Supervision
- 以分子片段为单位进行编辑重建,引入片段级监督信号
- 在多个下游任务中达到顶尖性能,超越部分3D表示模型
- 适合需要精细分子结构理解的药物发现与生成任务
SMILES作为分子结构的重要文本表示,已成为预训练语言模型(LM)的基础。然而,现有预训练SMILES LM仅在单标记层面进行监督,未能充分挖掘分子的子结构信息,导致预训练任务过于简单,难以捕捉更丰富的分子语义。此外,预训练时仅处理被破坏的SMILES输入,从未接触有效SMILES,造成训练-推理不一致。为此,我们提出SMI-Editor,一种基于编辑的预训练SMILES LM。该模型随机破坏分子中的子结构,并将生成的SMILES送回模型,使其通过编辑过程恢复原始结构。这一方法不仅引入了片段级训练信号,还允许使用有效SMILES作为输入,使模型学习从不完整结构重建完整分子。实验表明,SMI-Editor在多个下游分子任务中表现卓越,甚至优于多个3D分子表示模型。
原文摘要 · Abstract (English)
SMILES, a crucial textual representation of molecular structures, has garnered significant attention as a foundation for pre-trained language models (LMs). However, most existing pre-trained SMILES LMs focus solely on the single-token level supervision during pre-training, failing to fully leverage the substructural information of molecules. This limitation makes the pre-training task overly simplistic, preventing the models from capturing richer molecular semantic information. Moreover, during pre-training, these SMILES LMs only process corrupted SMILES inputs, never encountering any valid SMILES, which leads to a train-inference mismatch. To address these challenges, we propose SMI-Editor, a novel edit-based pre-trained SMILES LM. SMI-Editor disrupts substructures within a molecule at random and feeds the resulting SMILES back into the model, which then attempts to restore the original SMILES through an editing process. This approach not only introduces fragment-level training signals, but also enables the use of valid SMILES as inputs, allowing the model to learn how to reconstruct complete molecules from these incomplete structures. As a result, the model demonstrates improved scalability and an enhanced ability to capture fragment-level molecular information. Experimental results show that SMI-Editor achieves state-of-the-art performance across multiple downstream molecular tasks, and even outperforming several 3D molecular representation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。