arXiv:2511.06294cs.LGcs.AI2025-11AAAI被引 7

将物理注意力重构成线性注意力,提升PDE求解效率与性能。

Transolver is a Linear Transformer: Revisiting Physics-Attention through the Lens of Linear Attention

  • 用线性注意力重构物理注意力机制,剥离无关切片交互
  • 在6个标准PDE数据集上达最新性能,参数减少40%,计算量降36.2%
  • 工业级数据集上表现更优,适合高效高精度科学计算场景

基于Transformer的神经算子在数据驱动的偏微分方程(PDE)求解中取得显著进展。当前多数研究聚焦于降低注意力的二次复杂度以提升训练与推理效率。其中,Transolver通过引入物理注意力(Physics-Attention)实现降本:将网格点投影至切片进行切片注意力,再通过反切片映射回原空间。然而我们发现,物理注意力可被重新表述为一种特殊的线性注意力,且切片注意力甚至可能损害模型性能。据此,我们认为其有效性主要源于切片与反切片操作本身,而非切片间的交互。基于此洞察,我们提出两步变换,将物理注意力重构为标准线性注意力,命名为线性注意力神经算子(LinearNO)。该方法在六个标准PDE基准上达到当前最优性能,平均参数量减少40.0%,计算成本降低36.2%。同时,在两个挑战性的工业级数据集AirfRANS与Shape-Net Car上也展现出卓越表现。

原文摘要 · Abstract (English)

Recent advances in Transformer-based Neural Operators have enabled significant progress in data-driven solvers for Partial Differential Equations (PDEs). Most current research has focused on reducing the quadratic complexity of attention to address the resulting low training and inference efficiency. Among these works, Transolver stands out as a representative method that introduces Physics-Attention to reduce computational costs. Physics-Attention projects grid points into slices for slice attention, then maps them back through deslicing. However, we observe that Physics-Attention can be reformulated as a special case of linear attention, and that the slice attention may even hurt the model performance. Based on these observations, we argue that its effectiveness primarily arises from the slice and deslice operations rather than interactions between slices. Building on this insight, we propose a two-step transformation to redesign Physics-Attention into a canonical linear attention, which we call Linear Attention Neural Operator (LinearNO). Our method achieves state-of-the-art performance on six standard PDE benchmarks, while reducing the number of parameters by an average of 40.0% and computational cost by 36.2%. Additionally, it delivers superior performance on two challenging, industrial-level datasets: AirfRANS and Shape-Net Car.

PDE求解线性注意力神经算子高效建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。