改进物理信息神经网络，用傅里叶特征和注意力机制提升精度与效率。
Physics-Informed Neural Networks with Fourier Features and Attention-Driven Decoding
- 去除非必要编码器，仅靠自注意力捕捉时空相关性。
- 引入傅里叶特征嵌入，有效缓解频谱偏差问题。
- 在多个基准上优于原有架构，参数更少且性能更强。
物理信息神经网络（PINNs）是利用深度学习方法近似偏微分方程解的有力框架。本文提出对基于Transformer的PINN架构PINNSformer进行系统性重构，设计了谱域PINNSformer（S-Pformer）。该模型解决了两个关键问题：1）编码器冗余（即参数量增加）；2）频谱偏差缓解。研究发现，当仅依赖自注意力机制时，编码器对捕捉时空相关性并非必需，从而显著降低参数量。进一步地，通过引入傅里叶特征嵌入，显式缓解频谱偏差，实现频率域中多尺度行为的自适应编码。实验表明，S-Pformer在所有基准测试中均优于编码-解码结构的PINNSformer，性能达到甚至超过MLP水平，同时大幅减少参数量。
原文摘要 · Abstract (English)
Physics-Informed Neural Networks (PINNs) are a useful framework for approximating partial differential equation solutions using deep learning methods. In this paper, we propose a principled redesign of the PINNsformer, a Transformer-based PINN architecture. We present the Spectral PINNSformer (S-Pformer), a refinement of encoder-decoder PINNSformers that addresses two key issues; 1. the redundancy (i.e. increased parameter count) of the encoder, and 2. the mitigation of spectral bias. We find that the encoder is unnecessary for capturing spatiotemporal correlations when relying solely on self-attention, thereby reducing parameter count. Further, we integrate Fourier feature embeddings to explicitly mitigate spectral bias, enabling adaptive encoding of multiscale behaviors in the frequency domain. Our model outperforms encoder-decoder PINNSformer architectures across all benchmarks, achieving or outperforming MLP performance while reducing parameter count significantly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。