arXiv:2603.27929cs.LGcs.AI2026-03被引 1

将物理规律融入注意力机制,提升稀疏数据下科学建模的准确性与稳定性。

Physics-Guided Transformer (PGT): Physics-Aware Attention Mechanism for PINNs

  • 在注意力计算中加入热核偏置,显式编码扩散与时间因果关系
  • 1D重建相对误差仅5.9e-3,2D流场同时实现低残差与高精度
  • 适合物理方程约束强、数据稀疏的科学计算场景

从稀疏不规则观测重建连续物理场是科学机器学习的核心挑战,尤其针对由偏微分方程(PDE)支配的系统。现有物理信息方法通常将控制方程作为优化中的软惩罚项,常导致梯度失衡、不稳定及数据有限时物理一致性下降。我们提出物理引导的Transformer(PGT),一种将物理结构直接嵌入自注意力机制的神经架构。具体地,PGT在注意力logits中引入基于热核的加性偏置,编码扩散动态与时间因果性。查询坐标关注这些物理条件化的上下文标记,解码器采用FiLM调制的正弦隐式网络,自适应调控频谱响应。我们在一维热方程与二维不可压缩纳维-斯托克斯系统上评估PGT。在100个观测点的稀疏1D重建中,PGT相对L2误差为5.9e-3,显著优于PINNs与正弦表示。在二维圆柱尾流问题中,PGT唯一实现低PDE残差(8.3e-4)与竞争力相对误差(0.034),优于仅优化单一目标的方法。结果表明,在注意力中嵌入物理可提升数据稀缺下的稳定性、泛化性与物理保真度。

原文摘要 · Abstract (English)

Reconstructing continuous physical fields from sparse, irregular observations is a central challenge in scientific machine learning, particularly for systems governed by partial differential equations (PDEs). Existing physics-informed methods typically enforce governing equations as soft penalty terms during optimization, often leading to gradient imbalance, instability, and degraded physical consistency under limited data. We introduce the Physics-Guided Transformer (PGT), a neural architecture that embeds physical structure directly into the self-attention mechanism. Specifically, PGT incorporates a heat-kernel-derived additive bias into attention logits, encoding diffusion dynamics and temporal causality within the representation. Query coordinates attend to these physics-conditioned context tokens, and the resulting features are decoded using a FiLM-modulated sinusoidal implicit network that adaptively controls spectral response. We evaluate PGT on the one-dimensional heat equation and two-dimensional incompressible Navier-Stokes systems. In sparse 1D reconstruction with 100 observations, PGT achieves a relative L2 error of 5.9e-3, significantly outperforming both PINNs and sinusoidal representations. In the 2D cylinder wake problem, PGT uniquely achieves both low PDE residual (8.3e-4) and competitive relative error (0.034), outperforming methods that optimize only one objective. These results demonstrate that embedding physics within attention improves stability, generalization, and physical fidelity under data-scarce conditions.

物理信息注意力机制偏微分方程稀疏重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。