用傅里叶谱方法替代自动微分,训练物理神经网络更快更省内存。
A Computational Comparison of Fourier Spectral Differentiation and Spatial Automatic Differentiation in Periodic Physics-Informed Neural Networks

- 在周期性问题中用傅里叶变换计算空间导数,避免重复求导
- 训练速度提升2.9到18.5倍,显存占用减少68.7%~94.1%
- 适合需要高频导数或大模型的周期性物理问题
物理信息神经网络(PINNs)通常使用自动微分(AD)计算偏微分方程残差中的空间导数,当涉及多阶或高阶导数时,计算与内存开销会显著增加。本文在周期性物理空间的PINNs中,对空间自动微分与傅里叶谱微分进行了受控对比。每组实验中,神经网络结构、时间微分、优化器、采样方式和训练调度均保持一致,仅改变空间微分方法。傅里叶变体将网络输出在均匀周期网格上评估后转至傅里叶空间,通过谱乘法获取空间导数,并复用相同傅里叶系数以支持多阶导数。在标准PINNs的Allen-Cahn与Korteweg-de Vries方程,以及因果PINNs的Allen-Cahn、Korteweg-de Vries和Kuramoto-Sivashinsky方程上进行测试。在五种方程-框架组合中,傅里叶微分使端到端训练速度平均提升2.90×至18.52×,峰值GPU内存降低68.7%~94.1%。最终相对L₂误差处于同一数量级,两种方法无明显精度优劣。对于所考虑的一维周期性基准问题,傅里叶谱微分在保持相近解误差的同时,显著降低训练时间和内存消耗,代价是需使用均匀结构化空间网格。
原文摘要 · Abstract (English)
Physics-informed neural networks (PINNs) commonly evaluate the spatial derivatives appearing in partial differential equation residuals using automatic differentiation (AD), whose computational and memory costs can become substantial when multiple or high-order derivatives are required. We perform a controlled comparison of spatial AD and Fourier spectral differentiation in periodic physical-space PINNs. Within each paired experiment, the neural representation, temporal differentiation, optimizer, sampling procedure, and training schedule are held fixed, so that the two cases differ only in the spatial differentiation procedure. For the Fourier variant, network outputs are evaluated on a uniform periodic grid and transformed to Fourier space, where spatial derivatives are obtained through spectral multiplication and the same Fourier coefficients are reused across derivative orders. We compare the two procedures in standard PINNs for the Allen--Cahn and Korteweg--de Vries equations and in Causal PINNs for the Allen--Cahn, Korteweg--de Vries, and Kuramoto--Sivashinsky equations. Across these five equation--framework settings, Fourier differentiation yields mean paired end-to-end training speedups ranging from $2.90\times$ to $18.52\times$ and reduces peak allocated graphics processing unit (GPU) memory by $68.7\%$--$94.1\%$. The final relative $L_2$ errors remain of the same order, with neither differentiation procedure showing a consistent accuracy advantage. For the one-dimensional periodic benchmarks considered here, Fourier spectral differentiation therefore provides substantially lower training time and memory usage than spatial AD while retaining comparable solution error, at the cost of requiring a uniform structured spatial grid.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。