用函数投影法让Transformer更懂物理场,预测精度和泛化能力双提升。
Physics Transformer: Tailoring Transformer for General PDE Prediction

- 将物理场视为连续函数,动态生成自适应基函数来构建物理语义令牌。
- 在二维到三维流体模拟中均达到领先性能,能捕捉精细物理结构。
- 适合需要高精度物理建模的科研与工业仿真场景。
Transformer架构因其对不规则离散化和长程物理依赖的处理能力,正被广泛用于求解偏微分方程(PDE)。然而,物理场是无限维函数的有限采样,传统Tokenize方式难以保留其函数本质。为此,我们提出Physics Transformer,一种基于函数投影的Transformer架构。该方法将物理场视为连续函数,将其离散化划分为保局部性的空间块,在每个块内动态学习一组自适应局部基函数,并将采样场投影到这些基上以生成紧凑的物理令牌。所得表示既捕捉多样的潜在物理状态,又保留细粒度空间结构,通过空间与物理状态的因子化注意力实现高效全局交互。投影表示还支持在任意查询位置进行高效解码。在涵盖二维PDE动力学到三维工业级计算流体模拟的多种基准测试中,Physics Transformer准确捕捉了细粒度物理结构,实现了当前最优的预测性能。结果表明,函数投影是设计PDE求解Transformer的有效基础。
原文摘要 · Abstract (English)
Transformer architectures have attracted increasing attention for solving partial differential equations (PDEs), owing to their flexibility in handling irregular discretizations and their ability to capture long-range physical dependencies. However, unlike discrete language tokens or fixed-resolution image patches, observed physical fields are finite samples of underlying infinite-dimensional functions. Consequently, effectively applying Transformers to PDEs requires a tokenizer that respects the functional nature of physical fields and constructs physically expressive tokens from arbitrary discretizations.To this end, we propose \methodname{Physics Transformer}, a function-projection-based Transformer architecture for physical field prediction. Physics Transformer treats a physical field as a continuous function and partitions its discretization into locality-preserving spatial patches. Within each patch, it dynamically learns a set of adaptive local basis functions and projects the sampled field onto these bases to obtain compact physics tokens. The resulting tokens capture diverse latent physical states while preserving fine-scale spatial structures, enabling efficient global interaction through factorized attention across space and physical states. The projected representation further supports efficient decoding at arbitrary query locations. Extensive experiments on diverse benchmarks, ranging from two-dimensional PDE dynamics to industrial-scale three-dimensional CFD simulations, demonstrate that Physics Transformer accurately captures fine-grained physical structures and achieves state-of-the-art predictive performance. These results establish function projection as a practical and effective foundation for designing Transformer architectures for PDE solving.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。