提升神经算子对局部高频信号的建模能力,解决传统方法频谱偏差问题。
LOGLO-FNO: Efficient Learning of Local and Global Features in Fourier Neural Operators
- 引入局部谱卷积并行分支与高频传播模块,增强对非主导频率的捕捉。
- 在6个复杂PDE任务中优于现有神经算子模型,长时滚动更稳定。
- 设计径向分桶频谱损失,精准引导高频成分学习,参数量减少50%。
在科学机器学习中,建模高频信息是一项关键挑战。例如,雷诺数3500以上的纳维-斯托克斯方程全湍流模拟会因涡旋和漩涡引起的流体运动产生高频信号。神经网络准确重建中高频成分的能力取决于其对这些频率的建模精度。然而,神经网络存在固有的频谱偏置,倾向于学习低频成分。尽管傅里叶神经算子(FNOs)在多个偏微分方程基准测试中表现优异,但在学习由局部特征表征的非主导频率方面仍表现不佳。这一局限源于神经网络的固有频谱偏置,以及FNO及其变体显式排除高频模式的问题。为此,本文提出两项关键架构改进:(i) 并行分支进行局部谱卷积;(ii) 高频传播模块。此外,提出一种基于径向分桶谱误差的新型频率敏感损失函数。该并行分支使可训练参数最多减少50%,同时达到仅使用全局卷积的FNO的精度。实验在六个流体力学、波传播和生物模式形成中的挑战性PDE上进行,定性和谱分析表明,本方法在性能上显著优于当前主流神经算子基线模型。
原文摘要 · Abstract (English)
Modeling high-frequency information is a critical challenge in scientific machine learning. For instance, fully turbulent flow simulations of the Navier-Stokes equations at Reynolds numbers 3500 and above can generate high-frequency signals due to swirling fluid motions caused by eddies and vortices. Faithfully modeling such signals using neural nets depends on the accurate reconstruction of moderate to high frequencies. However, it has been well known that neural nets exhibit spectral or frequency bias towards learning low-frequency components. Meanwhile, Fourier Neural Operators (FNOs) have emerged as a popular class of data-driven models for surrogate modeling and solving PDEs. Although impressive results were achieved on several PDE benchmark problems, FNOs perform poorly in learning non-dominant frequencies characterized by local features. This limitation stems from spectral bias inherent in neural nets and the explicit exclusion of high-frequency modes in FNOs and their variants. Therefore, to mitigate these issues and improve FNO's spectral learning capabilities to represent a broad range of frequency components, we propose two key architectural enhancements: (i) a parallel branch performing local spectral convolution (ii) a high-frequency propagation module. Moreover, we propose a novel frequency-sensitive loss based on radially binned spectral errors. This introduction of a parallel branch for local convolution reduces the trainable parameters by up to 50% while achieving the accuracy of FNO that relies solely on global convolution. Moreover, our findings demonstrate that the proposed model improves stability over longer rollouts. Experiments on six challenging PDEs in fluid mechanics, wave propagation, and biological pattern formation, and the qualitative and spectral analysis of predictions, show the effectiveness of our method over SOTA neural operator families of baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。