通过低频预训练+高频微调，提升神经PDE求解器精度与效率。
FreqMoE: Dynamic Frequency Enhancement for Neural PDE Solvers
- 基于频率依赖性构建频域混合专家模型，动态扩展高频权重。
- 相比密集FNO，参数减少47.32倍（仅2.1%），精度提升最高16.6%。
- 适用于高分辨率输入与长期预测，兼容多种网格结构和FNO变体。
傅里叶神经算子（FNO）通过频域变换学习无限维函数映射，成为高效求解偏微分方程（PDE）的有前景方法。然而，高频信号稀疏导致高维输入下计算效率低，固定截断模式常造成高频信息丢失，影响高分辨率输入或长时预测性能。为此，我们提出FreqMoE，一种高效渐进式训练框架，利用高频信号对低频成分的依赖关系。模型先学习低频权重，再采用稀疏上行循环策略，在频域构建混合专家（MoE），有效将学习到的权重扩展至高频区域。在规则与不规则网格PDE上的实验表明，FreqMoE相较密集FNO实现最高16.6%精度提升，参数仅占2.1%（减少47.32倍）。此外，该方法在长期预测中表现出显著稳定性，并可无缝推广至多种FNO变体与网格结构，建立新的“低频预训练，高频微调”求解范式。
原文摘要 · Abstract (English)
Fourier Neural Operators (FNO) have emerged as promising solutions for efficiently solving partial differential equations (PDEs) by learning infinite-dimensional function mappings through frequency domain transformations. However, the sparsity of high-frequency signals limits computational efficiency for high-dimensional inputs, and fixed-pattern truncation often causes high-frequency signal loss, reducing performance in scenarios such as high-resolution inputs or long-term predictions. To address these challenges, we propose FreqMoE, an efficient and progressive training framework that exploits the dependency of high-frequency signals on low-frequency components. The model first learns low-frequency weights and then applies a sparse upward-cycling strategy to construct a mixture of experts (MoE) in the frequency domain, effectively extending the learned weights to high-frequency regions. Experiments on both regular and irregular grid PDEs demonstrate that FreqMoE achieves up to 16.6% accuracy improvement while using merely 2.1% parameters (47.32x reduction) compared to dense FNO. Furthermore, the approach demonstrates remarkable stability in long-term predictions and generalizes seamlessly to various FNO variants and grid structures, establishing a new ``Low frequency Pretraining, High frequency Fine-tuning'' paradigm for solving PDEs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。