用注意力机制建模物理演化过程,实现跨时空预测与未知环境适应。
An Attention-based Spatio-Temporal Neural Operator for Evolving Physics
- 分离式注意力捕捉时空交互,结合后向微分公式设计时间预测模块。
- 在多个科学计算基准上超越现有模型,尤其在未知环境泛化性能突出。
- 适合需要可解释性与物理规律发现的工程仿真与科学建模场景。
在科学机器学习中,核心挑战在于学习未知且动态演化的物理过程,并实现跨时空尺度的预测。例如,在增材制造等实际制造问题中,用户调整已知机器参数的同时,未知环境参数也在持续波动。为实现可靠预测,模型需既能从数据中捕捉长程时空关联,又能适应新出现的未知环境;传统机器学习模型虽擅长前者,却常缺乏物理可解释性,且在环境变化时泛化能力差。为此,我们提出注意力驱动的时空神经算子(ASNO),通过分离的空间与时间注意力机制,实现对未知物理参数的自适应。受后向微分公式(BDF)启发,ASNO构建了用于时间预测与外推的Transformer结构,并采用基于注意力的神经算子处理变化的外部载荷,通过解耦历史状态贡献与外部力作用,提升可解释性,助力发现底层物理规律,并实现对未见物理环境的泛化。在多个科学机器学习基准上的实验表明,ASNO显著优于现有模型,展现出在工程应用、物理发现与可解释机器学习中的潜力。
原文摘要 · Abstract (English)
In scientific machine learning (SciML), a key challenge is learning unknown, evolving physical processes and making predictions across spatio-temporal scales. For example, in real-world manufacturing problems like additive manufacturing, users adjust known machine settings while unknown environmental parameters simultaneously fluctuate. To make reliable predictions, it is desired for a model to not only capture long-range spatio-temporal interactions from data but also adapt to new and unknown environments; traditional machine learning models excel at the first task but often lack physical interpretability and struggle to generalize under varying environmental conditions. To tackle these challenges, we propose the Attention-based Spatio-Temporal Neural Operator (ASNO), a novel architecture that combines separable attention mechanisms for spatial and temporal interactions and adapts to unseen physical parameters. Inspired by the backward differentiation formula (BDF), ASNO learns a transformer for temporal prediction and extrapolation and an attention-based neural operator for handling varying external loads, enhancing interpretability by isolating historical state contributions and external forces, enabling the discovery of underlying physical laws and generalizability to unseen physical environments. Empirical results on SciML benchmarks demonstrate that ASNO outperforms over existing models, establishing its potential for engineering applications, physics discovery, and interpretable machine learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。