通过建模分子周围三维空间,提升药物发现中的预训练分子表示性能
Beyond Atoms: Enhancing Molecular Pretrained Representations with 3D Space Modeling
- 将分子周围3D空间离散化为网格,用Transformer架构学习空间信息
- 在多个下游任务中超越现有3D分子模型,在数据有限时表现更优
- 适合需要高精度分子表征的药物设计与材料研发场景
分子预训练表示(MPR)已成为解决药物发现与材料设计中监督数据稀缺问题的强大方法。早期方法依赖一维序列和二维图结构,近年研究引入三维构象信息以捕捉原子间复杂相互作用。然而,这些模型仍将分子视为离散原子集合,忽略了原子周围的三维空间。本文从物理视角指出,仅建模原子点是不足的。我们首次观察到:在原子外随机采样虚拟点即可显著提升MPR性能。基于此,提出一种系统性框架,显式建模分子所占据的完整三维空间。实现上采用新型Transformer架构SpaceFormer,包含三个核心组件:(1) 基于网格的三维空间离散化;(2) 网格采样与合并机制;(3) 高效的3D位置编码。大量实验表明,SpaceFormer在多个下游任务中显著优于现有3D MPR模型,验证了利用原子之外三维空间对提升分子表示的有效性。
原文摘要 · Abstract (English)
Molecular pretrained representations (MPR) has emerged as a powerful approach for addressing the challenge of limited supervised data in applications such as drug discovery and material design. While early MPR methods relied on 1D sequences and 2D graphs, recent advancements have incorporated 3D conformational information to capture rich atomic interactions. However, these prior models treat molecules merely as discrete atom sets, overlooking the space surrounding them. We argue from a physical perspective that only modeling these discrete points is insufficient. We first present a simple yet insightful observation: naively adding randomly sampled virtual points beyond atoms can surprisingly enhance MPR performance. In light of this, we propose a principled framework that incorporates the entire 3D space spanned by molecules. We implement the framework via a novel Transformer-based architecture, dubbed SpaceFormer, with three key components: (1) grid-based space discretization; (2) grid sampling/merging; and (3) efficient 3D positional encoding. Extensive experiments show that SpaceFormer significantly outperforms previous 3D MPR models across various downstream tasks with limited data, validating the benefit of leveraging the additional 3D space beyond atoms in MPR models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。