arXiv:2412.05569cs.LGq-bio.BM2024-12ICLR被引 8

通过片段编辑训练分子SMILES模型,提升结构理解能力

SMI-Editor: Edit-based SMILES Language Model with Fragment-level Supervision

  • 以分子片段为单位进行编辑重建,引入片段级监督信号
  • 在多个下游任务中达到顶尖性能,超越部分3D表示模型
  • 适合需要精细分子结构理解的药物发现与生成任务

SMILES作为分子结构的重要文本表示,已成为预训练语言模型(LM)的基础。然而,现有预训练SMILES LM仅在单标记层面进行监督,未能充分挖掘分子的子结构信息,导致预训练任务过于简单,难以捕捉更丰富的分子语义。此外,预训练时仅处理被破坏的SMILES输入,从未接触有效SMILES,造成训练-推理不一致。为此,我们提出SMI-Editor,一种基于编辑的预训练SMILES LM。该模型随机破坏分子中的子结构,并将生成的SMILES送回模型,使其通过编辑过程恢复原始结构。这一方法不仅引入了片段级训练信号,还允许使用有效SMILES作为输入,使模型学习从不完整结构重建完整分子。实验表明,SMI-Editor在多个下游分子任务中表现卓越,甚至优于多个3D分子表示模型。

原文摘要 · Abstract (English)

SMILES, a crucial textual representation of molecular structures, has garnered significant attention as a foundation for pre-trained language models (LMs). However, most existing pre-trained SMILES LMs focus solely on the single-token level supervision during pre-training, failing to fully leverage the substructural information of molecules. This limitation makes the pre-training task overly simplistic, preventing the models from capturing richer molecular semantic information. Moreover, during pre-training, these SMILES LMs only process corrupted SMILES inputs, never encountering any valid SMILES, which leads to a train-inference mismatch. To address these challenges, we propose SMI-Editor, a novel edit-based pre-trained SMILES LM. SMI-Editor disrupts substructures within a molecule at random and feeds the resulting SMILES back into the model, which then attempts to restore the original SMILES through an editing process. This approach not only introduces fragment-level training signals, but also enables the use of valid SMILES as inputs, allowing the model to learn how to reconstruct complete molecules from these incomplete structures. As a result, the model demonstrates improved scalability and an enhanced ability to capture fragment-level molecular information. Experimental results show that SMI-Editor achieves state-of-the-art performance across multiple downstream molecular tasks, and even outperforming several 3D molecular representation models.

分子生成SMILES预训练模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。