arXiv:2603.03112cs.LGcs.AI2026-03被引 1

将物理动力学融入Transformer,高效建模多尺度偏微分方程。

From Complex Dynamics to DynFormer: Rethinking Transformers for PDEs

  • 按物理尺度分离设计网络模块,避免全局注意力冗余计算。
  • 在4个基准上相对误差降低95%,显存消耗显著减少。
  • 适合需要高精度与低资源的科学计算场景使用。

偏微分方程(PDE)是建模复杂物理系统的基础,但传统数值求解器在高维和多尺度场景下计算成本过高。基于Transformer的神经算子虽为数据驱动替代方案,但通常将所有离散空间点视为均质独立的标记,忽略物理场的固有尺度分离,导致计算量巨大的全局注意力机制冗余混合平滑大尺度动态与高频波动。本文从复杂动力学视角重构Transformer,提出DynFormer——一种动力学感知的神经算子。它不采用统一注意力机制,而是为不同物理尺度分配专用模块:通过谱嵌入分离低频模态,实现克罗内克结构注意力,高效捕捉大尺度全局相互作用;同时引入局部-全局混合变换,利用非线性乘法频率混合作用,隐式重建受宏观状态支配的小尺度湍流级联,无需全局注意力开销。集成于混合演化架构中,确保长期时间稳定性。在四个PDE基准上的内存对齐评估表明,相比现有最优基线,DynFormer相对误差最高降低95%,且显著降低GPU显存占用。结果表明,将第一性原理物理动力学嵌入Transformer架构,可为PDE代理建模提供高可扩展、理论严谨的新范式。

原文摘要 · Abstract (English)

Partial differential equations (PDEs) are fundamental for modeling complex physical systems, yet classical numerical solvers face prohibitive computational costs in high-dimensional and multi-scale regimes. While Transformer-based neural operators have emerged as powerful data-driven alternatives, they conventionally treat all discretized spatial points as uniform, independent tokens. This monolithic approach ignores the intrinsic scale separation of physical fields, applying computationally prohibitive global attention that redundantly mixes smooth large-scale dynamics with high-frequency fluctuations. Rethinking Transformers through the lens of complex dynamics, we propose DynFormer, a novel dynamics-informed neural operator. Rather than applying a uniform attention mechanism across all scales, DynFormer explicitly assigns specialized network modules to distinct physical scales. It leverages a Spectral Embedding to isolate low-frequency modes, enabling a Kronecker-structured attention mechanism to efficiently capture large-scale global interactions with reduced complexity. Concurrently, we introduce a Local-Global-Mixing transformation. This module utilizes nonlinear multiplicative frequency mixing to implicitly reconstruct the small-scale, fast-varying turbulent cascades that are slaved to the macroscopic state, without incurring the cost of global attention. Integrating these modules into a hybrid evolutionary architecture ensures robust long-term temporal stability. Extensive memory-aligned evaluations across four PDE benchmarks demonstrate that DynFormer achieves up to a 95% reduction in relative error compared to state-of-the-art baselines, while significantly reducing GPU memory consumption. Our results establish that embedding first-principles physical dynamics into Transformer architectures yields a highly scalable, theoretically grounded blueprint for PDE surrogate modeling.

偏微分方程Transformer神经算子多尺度建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。