提出新型谱注意力网络，更准求解偏微分方程。
SAOT: An Enhanced Locality-Aware Spectral Transformer for Solving PDEs
- 用小波注意力模块捕捉局部细节与高频特征
- 在6个基准上达到当前最优性能，且对网格不敏感
- 适合需要高精度解的科学计算场景
神经算子在求解一类偏微分方程（PDEs）方面展现出巨大潜力，通过建模输入与输出函数之间的映射关系实现。傅里叶神经算子（FNO）通过参数化傅里叶空间中的积分算子来实现全局卷积，但常导致解过度平滑，难以捕捉局部细节和高频成分。为解决此问题，本文将小波变换的空间-频率局部性特性引入Transformer架构，提出一种计算复杂度为线性的新式小波注意力（WA）模块，可高效学习局部感知特征。基于WA，进一步构建了谱注意力算子变压器（SAOT），一种融合小波局部聚焦与傅里叶注意力全局感受野的混合谱Transformer框架，通过门控融合块实现特征整合。实验表明，WA显著缓解了传统傅里叶注意力的缺陷，在多个基准上优于现有小波基神经算子。通过结合局部感知与全局频域表示，SAOT在六个算子学习基准上取得当前最优表现，并展现出强大的离散化不变能力。
原文摘要 · Abstract (English)
Neural operators have shown great potential in solving a family of Partial Differential Equations (PDEs) by modeling the mappings between input and output functions. Fourier Neural Operator (FNO) implements global convolutions via parameterizing the integral operators in Fourier space. However, it often results in over-smoothing solutions and fails to capture local details and high-frequency components. To address these limitations, we investigate incorporating the spatial-frequency localization property of Wavelet transforms into the Transformer architecture. We propose a novel Wavelet Attention (WA) module with linear computational complexity to efficiently learn locality-aware features. Building upon WA, we further develop the Spectral Attention Operator Transformer (SAOT), a hybrid spectral Transformer framework that integrates WA's localized focus with the global receptive field of Fourier-based Attention (FA) through a gated fusion block. Experimental results demonstrate that WA significantly mitigates the limitations of FA and outperforms existing Wavelet-based neural operators by a large margin. By integrating the locality-aware and global spectral representations, SAOT achieves state-of-the-art performance on six operator learning benchmarks and exhibits strong discretization-invariant ability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。