arXiv:2607.07718cs.LGcs.AI2026-07

提出局部线性Transformer,高效精准求解偏微分方程。

LLT: Local Linear Transformer for PDE Operator Learning

论文配图:LLT: Local Linear Transformer for PDE Operator Learning
图 1 · 摘自论文原文
  • 结合线性全局注意力与局部空间混合,引入坐标几何信息。
  • 在多种问题上相对误差低于或相当现有方法,训练速度提升1.8至2.5倍。
  • 适用于结构与非结构网格、不同离散化方式,适合大规模复杂仿真。

神经算子已成为学习偏微分方程(PDE)解映射和加速数值模拟的常用方法。基于Transformer的神经算子尤为受关注,因其注意力机制可捕捉计算域中的长程依赖关系。然而,标准注意力在应用于PDE时存在两大缺陷:随计算节点数呈二次增长,且缺乏对局部相互作用的显式偏好。为此,本文提出局部线性Transformer(LLT)用于PDE算子学习。该架构融合线性全局注意力与局部空间混合,并融入坐标与几何信息。我们在多个PDE问题上评估了LLT,包括弹性、塑性、机翼气流、管道流动及达西流。参考数据涵盖有限元、有限体积和有限差分离散化,涉及结构与非结构网格。相比先前研究中的其他神经算子与Transformer基线,LLT在这些问题上实现了竞争性或更低的相对$L_2$误差。在匹配的结构化离散化下,每轮训练的壁时钟时间比Transolver减少1.8至2.5倍。我们还将该方法扩展至包含32,186个非结构网格点的三维汽车气动数据集。综合结果表明,LLT在不同离散化、网格类型和问题设置下均能提供准确且高效的PDE算子。

原文摘要 · Abstract (English)

Neural operators have become a common approach for learning PDE solution maps and accelerating numerical simulations. Transformer-based neural operators are of particular interest, since attention can learn long-range dependencies in the computational domain. However, standard attention has two major limitations when applied to PDEs: it scales quadratically with the number of computational nodes, and it lacks an explicit bias toward local interactions. To address these issues, we introduce Local Linear Transformer (LLT) for PDE operator learning. The architecture combines linear global attention with local spatial mixing, and incorporates coordinate and geometry information. We evaluate LLT on several PDE problems, including elasticity, plasticity, airfoil flow, pipe flow, and Darcy flow. The reference data for these problems span finite-element, finite-volume, and finite-difference discretizations on structured and unstructured meshes. Compared with other neural-operator and transformer baselines from prior studies, LLT achieves competitive or lower relative $L_2$ error across these problems. On matched structured discretizations, wall-clock time per training iteration is reduced by factors of 1.8 to 2.5 relative to Transolver. We also scale the approach and apply it to a three-dimensional car aerodynamics dataset with 32,186 unstructured mesh points per sample. Together, these results indicate that LLT provides an accurate and computationally efficient operator for PDE problems across discretizations, mesh types, and problem settings.

PDE求解Transformer神经算子高效建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。