用新型Transformer提升网格物理模拟的效率与规模,最快比现有方法快7倍。
Training Transformers for Mesh-Based Simulations
- 以邻接矩阵为注意力掩码,结合扩张滑窗和全局注意力扩展感受野。
- 模型可处理30万节点、300万边的大规模网格,最小模型速度提升7倍。
- 适合做大规模物理仿真、高精度流体模拟的研究者使用。
基于图神经网络(GNN)的物理模拟主要依赖消息传递架构,但在处理大规模复杂网格时存在扩展性和效率问题。尽管已有多种改进如多尺度方法和K跳聚合,但通常引入显著复杂性且缺乏深入研究。为此,本文提出一种新型图Transformer架构,将邻接矩阵作为注意力掩码,并引入扩张滑窗与全局注意力机制,在不牺牲计算效率的前提下扩大感受野。通过在具有挑战性的三维流体动力学(CFD)数据集上进行大量实验,评估了模型规模、邻接矩阵增强、位置编码及K跳配置的影响。训练超过60个模型以发现训练浮点运算量(FLOPs)与参数量之间的缩放规律。所提模型展现出卓越的可扩展性,可在包含最多30万节点、300万边的网格上运行。最小模型性能媲美MeshGraphNet,但速度更快7倍、体积小6倍;最大模型平均性能超越此前最优水平38.8%,在全部回滚均方根误差(all-rollout RMSE)上领先MeshGraphNet达52%,训练速度相近。代码与数据集见https://github.com/DonsetPG/graph-physics。
原文摘要 · Abstract (English)
Simulating physics using Graph Neural Networks (GNNs) is predominantly driven by message-passing architectures, which face challenges in scaling and efficiency, particularly in handling large, complex meshes. These architectures have inspired numerous enhancements, including multigrid approaches and $K$-hop aggregation (using neighbours of distance $K$), yet they often introduce significant complexity and suffer from limited in-depth investigations. In response to these challenges, we propose a novel Graph Transformer architecture that leverages the adjacency matrix as an attention mask. The proposed approach incorporates innovative augmentations, including Dilated Sliding Windows and Global Attention, to extend receptive fields without sacrificing computational efficiency. Through extensive experimentation, we evaluate model size, adjacency matrix augmentations, positional encoding and $K$-hop configurations using challenging 3D computational fluid dynamics (CFD) datasets. We also train over 60 models to find a scaling law between training FLOPs and parameters. The introduced models demonstrate remarkable scalability, performing on meshes with up to 300k nodes and 3 million edges. Notably, the smallest model achieves parity with MeshGraphNet while being $7\times$ faster and $6\times$ smaller. The largest model surpasses the previous state-of-the-art by $38.8$\% on average and outperforms MeshGraphNet by $52$\% on the all-rollout RMSE, while having a similar training speed. Code and datasets are available at https://github.com/DonsetPG/graph-physics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。