提出SUPRA,让神经算子在不规则域上更准更快。
SUPRA: Subspace Parameterized Attention for Neural Operator on General Domains
- 用子空间参数化注意力,把无限维函数空间运算变有限维近似
- 在多个PDE数据集上误差降低最高达33%,计算效率领先
- 适合做复杂几何下偏微分方程求解的科研人员
神经算子是求解偏微分方程(PDE)的高效代理模型,但其核心组件面临挑战:(1) 为提升精度,标准注意力机制在大规模网格上计算低效;(2) 谱卷积依赖于规则网格上的快速傅里叶变换(FFT),且假设几何平坦,导致在不规则域上精度下降。为此,我们将欧氏空间中向量的矩阵-向量运算视为双线性型与线性算子,并将其推广至函数空间。新注意力机制在形式上等价于标准注意力,但因函数空间无限维而无法直接计算。为此,受模型降阶技术启发,提出子空间参数化注意力(SUPRA)神经算子,通过在有限维子空间中近似注意力机制。为在不规则域构建子空间,提出使用拉普拉斯特征函数,其天然适配域几何并保证对光滑函数的最优逼近。实验表明,SUPRA在多种PDE数据集上将误差降低最高达33%,同时保持最先进的计算效率。
原文摘要 · Abstract (English)
Neural operators are efficient surrogate models for solving partial differential equations (PDEs), but their key components face challenges: (1) in order to improve accuracy, attention mechanisms suffer from computational inefficiency on large-scale meshes, and (2) spectral convolutions rely on the Fast Fourier Transform (FFT) on regular grids and assume a flat geometry, which causes accuracy degradation on irregular domains. To tackle these problems, we regard the matrix-vector operations in the standard attention mechanism on vectors in Euclidean space as bilinear forms and linear operators in vector spaces and generalize the attention mechanism to function spaces. This new attention mechanism is fully equivalent to the standard attention but impossible to compute due to the infinite dimensionality of function spaces. To address this, inspired by model reduction techniques, we propose a Subspace Parameterized Attention (SUPRA) neural operator, which approximates the attention mechanism within a finite-dimensional subspace. To construct a subspace on irregular domains for SUPRA, we propose using the Laplacian eigenfunctions, which naturally adapt to domains' geometry and guarantee the optimal approximation for smooth functions. Experiments show that the SUPRA neural operator reduces error rates by up to 33% on various PDE datasets while maintaining state-of-the-art computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。