提出SEA注意力机制，显著降低物理系统预测的误差累积。
SEA: State-Exchange Attention for High-Fidelity Physics Based Transformers
- 用跨场多头注意力实现状态变量间双向信息交换
- 在动态系统预测中误差比基线降低88%至91%
- 特别适合依赖多状态耦合的复杂物理系统建模
现有基于序列网络的方法在估计动力系统场变量方面表现良好，但常受限于高滚动误差。由于误差积累问题，网络对未来预测越远，结果越不可靠，每一步的误差都会叠加导致精度下降。本文提出一种基于Transformer的State-Exchange Attention（SEA）模块，通过多头交叉注意力实现编码场之间的信息交换。该设计使系统中所有状态变量可相互传递信息，捕捉场间的物理关系与对称性。同时，引入类似ViT的网格自编码器，高效生成大量网格单元的空间一致嵌入。集成SEA的Transformer在滚动误差上达到当前最优表现：相比PbGMR-GMUS Transformer-RealNVP和GMR-GMUS Transformer，误差分别降低88%和91%。此外，仅使用SEA模块即可使高度依赖其他状态的状态变量误差减少97%。代码已开源。
原文摘要 · Abstract (English)
Current approaches using sequential networks have shown promise in estimating field variables for dynamical systems, but they are often limited by high rollout errors. The unresolved issue of rollout error accumulation results in unreliable estimations as the network predicts further into the future, with each step's error compounding and leading to an increase in inaccuracy. Here, we introduce the State-Exchange Attention (SEA) module, a novel transformer-based module enabling information exchange between encoded fields through multi-head cross-attention. The cross-field multidirectional information exchange design enables all state variables in the system to exchange information with one another, capturing physical relationships and symmetries between fields. Additionally, we introduce an efficient ViT-like mesh autoencoder to generate spatially coherent mesh embeddings for a large number of meshing cells. The SEA integrated transformer demonstrates the state-of-the-art rollout error compared to other competitive baselines. Specifically, we outperform PbGMR-GMUS Transformer-RealNVP and GMR-GMUS Transformer, with a reduction in error of 88% and 91%, respectively. Furthermore, we demonstrate that the SEA module alone can reduce errors by 97% for state variables that are highly dependent on other states of the system. The repository for this work is available at: https://github.com/ParsaEsmati/SEA
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。