对比四种模型,发现全局注意力对长程作用力更有效。
When Does Global Attention Help? A Unified Empirical Study on Atomistic Graph Learning
- 构建统一框架,控制变量比较消息传递与全局注意力
- 编码增强的MPNN表现稳健,融合模型在长程作用中更优
- 适合研究材料性质与分子建模的开发者参考
图神经网络(GNN)广泛用于替代高成本实验和第一性原理模拟,以研究原子尺度下化合物的行为。尽管最新GNN常结合传统消息传递网络(MPNN)与具备全局注意力机制的图变换器(GT)来建模短程与长程相互作用,但其优势仍不明确,因实现方式、特征或超参数调优差异导致结果不可比。本文提出首个统一可复现的基准测试框架——基于HydraGNN,支持无缝切换四种受控模型类别:纯MPNN、带化学/拓扑编码器的MPNN、MPNN与全局注意力的混合模型,以及融合局部-全局结构的全模型。在七个开源数据集上,涵盖回归与分类任务,系统分离了消息传递、全局注意力与编码器特征增强的贡献。结果表明,编码增强的MPNN构成稳健基线,而融合式局部-全局模型在受长程相互作用影响的性质预测中表现最佳。进一步量化了注意力机制的精度-计算权衡,报告其内存开销。本研究首次实现了原子图学习中全局注意力的受控评估,并为未来模型开发提供可复现的测试平台。
原文摘要 · Abstract (English)
Graph neural networks (GNNs) are widely used as surrogates for costly experiments and first-principles simulations to study the behavior of compounds at atomistic scale, and their architectural complexity is constantly increasing to enable the modeling of complex physics. While most recent GNNs combine more traditional message passing neural networks (MPNNs) layers to model short-range interactions with more advanced graph transformers (GTs) with global attention mechanisms to model long-range interactions, it is still unclear when global attention mechanisms provide real benefits over well-tuned MPNN layers due to inconsistent implementations, features, or hyperparameter tuning. We introduce the first unified, reproducible benchmarking framework - built on HydraGNN - that enables seamless switching among four controlled model classes: MPNN, MPNN with chemistry/topology encoders, GPS-style hybrids of MPNN with global attention, and fully fused local-global models with encoders. Using seven diverse open-source datasets for benchmarking across regression and classification tasks, we systematically isolate the contributions of message passing, global attention, and encoder-based feature augmentation. Our study shows that encoder-augmented MPNNs form a robust baseline, while fused local-global models yield the clearest benefits for properties governed by long-range interaction effects. We further quantify the accuracy-compute trade-offs of attention, reporting its overhead in memory. Together, these results establish the first controlled evaluation of global attention in atomistic graph learning and provide a reproducible testbed for future model development.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。