用多尺度序列化方法让模型更准预测分子性质
Multi-scale Graph Autoregressive Modeling: Molecular Property Prediction via Next Token Prediction
- 将分子图转为结构丰富的因果序列，分层次建模局部到全局
- 在MoleculeNet和MoleculeACE上超越现有方法，尤其擅长识别活性悬崖
- 适合药物研发中需要精准预测分子性质的研究者
我们提出连接感知基团序列（CamS），一种图到序列的表示方法，使仅解码器的Transformer可通过标准的下一个词预测（NTP）学习分子图。基于SMILES的NTP虽可扩展但缺乏显式拓扑，而原生图掩码建模虽捕捉连接性却可能破坏关键化学细节（如活性悬崖）。CamS通过数据驱动挖掘连接感知基团，并以骨架为中心的广度优先搜索序列化，建立稳定的中心到外围顺序。关键在于，它通过拼接细粒度到粗粒度基团序列实现层级建模，使模型能基于密集且未受损的局部结构证据来推断全局骨架。我们用原始LLaMA主干在CamS序列上预训练得到CamS-LLaMA，其在MoleculeNet和活性悬崖基准数据集MoleculeACE上达到当前最优性能，显著优于基于SMILES的语言模型和强图基基线。可解释性分析表明，多尺度因果序列有效引导注意力聚焦于决定悬崖差异的关键结构特征。
原文摘要 · Abstract (English)
We present Connection-Aware Motif Sequencing (CamS), a graph-to-sequence representation that enables decoder-only Transformers to learn molecular graphs via standard next-token prediction (NTP). For molecular property prediction, SMILES-based NTP scales well but lacks explicit topology, whereas graph-native masked modeling captures connectivity but risks disrupting the pivotal chemical details (e.g., activity cliffs). CamS bridges this gap by serializing molecular graphs into structure-rich causal sequences. CamS first mines data-driven connection-aware motifs. It then serializes motifs via scaffold-rooted breadth-first search (BFS) to establish a stable core-to-periphery order. Crucially, CamS enables hierarchical modeling by concatenating sequences from fine to coarse motif scales, allowing the model to condition global scaffolds on dense, uncorrupted local structural evidence. We instantiate CamS-LLaMA by pre-training a vanilla LLaMA backbone on CamS sequences. It achieves state-of-the-art performance on MoleculeNet and the activity-cliff benchmark MoleculeACE, outperforming both SMILES-based language models and strong graph baselines. Interpretability analysis confirms that our multi-scale causal serialization effectively drives attention toward cliff-determining differences.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。