用普通Transformer做分子动力学,不靠特殊设计也能达到顶尖效果
How simple can you go? An off-the-shelf transformer approach to molecular dynamics
- 用未修改的Edge Transformer架构,仅通过预训练处理分子结构
- 在3000万分子数据上预训练后,微调少量步骤即达领先性能
- 首次区分非协变误差与数值误差,适合追求简洁模型的研究者
当前多数分子动力学(MD)神经网络包含物理先验知识,导致架构复杂。本文挑战这一趋势,提出仅含极少领域特化特征的MD模型——MD-ET,基于未经修改的Edge Transformer架构,既无内置旋转协变性也无能量守恒设计。模型在约3000万分子结构的QCML数据库上进行简单监督预训练,微调数步后即在多个基准测试中取得领先结果。我们还分析了近似协变性和能量守恒对模拟的影响,提出新方法以区分非协变误差与其他误差(如数值舍入)。尽管大分子出现能量发散,但小分子体系仍可实现近似能量守恒的NVE模拟。
原文摘要 · Abstract (English)
Most current neural networks for molecular dynamics (MD) include physical inductive biases, resulting in specialized and complex architectures. This is in contrast to most other machine learning domains, where specialist approaches are increasingly replaced by general-purpose architectures trained on vast datasets. In line with this trend, several recent studies have questioned the necessity of architectural features commonly found in MD models, such as built-in rotational equivariance or energy conservation. In this work, we contribute to the ongoing discussion by evaluating the performance of an MD model with as few specialized architectural features as possible. We present a recipe for MD using an Edge Transformer, an ``off-the-shelf'' transformer architecture that has been minimally modified for the MD domain, termed MD-ET. Our model implements neither built-in equivariance nor energy conservation. We use a simple supervised pre-training scheme on $\sim$30 million molecular structures from the QCML database. Using this ``off-the-shelf'' approach, we show state-of-the-art results on several benchmarks after fine-tuning for a small number of steps. Additionally, we examine the effects of being only approximately equivariant and energy conserving for MD simulations, proposing a novel method for distinguishing the errors resulting from non-equivariance from other sources of inaccuracies like numerical rounding errors. While our model exhibits runaway energy increases on larger structures, we show approximately energy-conserving NVE simulations for a range of small structures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。