有限差分法可替代自动微分，在PINNs中更高效且节省显存。
Derivative Computation in PINNs: Automatic Differentiation, Finite Differences and Beyond
- 用合理步长的有限差分法替代自动微分，精度相当
- 在三种典型偏微分方程上，速度更快、显存消耗低一个数量级
- 对带样本间依赖结构的网络，有限差分能给出更接近真实梯度的近似
我们系统研究了物理信息神经网络（PINNs）中有限差分（FD）求导方法作为自动微分（AD）的替代方案。在三个基准偏微分方程问题上，我们证明：通过合理校准步长，FD在所有问题上精度与AD相当，同时在全部测试批量范围内运行更快、显存占用显著更低；我们提出的随机变体在稳态问题上表现优于AD。此外，我们发现对于具有样本间依赖的神经网络架构（如批归一化、自注意力），标准PyTorch autograd机制在隐式上是错误的；正确的逐样本计算在PINN常用批量规模下不可行，而有限差分提供的前向仅近似方法，实测结果比标准方法更接近真实逐样本导数一个数量级。
原文摘要 · Abstract (English)
We systematically investigate finite-difference (FD) derivative computation in Physics-Informed Neural Networks (PINNs) as an alternative to automatic differentiation (AD). On three benchmark PDEs we show that, with a properly calibrated step size, FD matches AD in accuracy on every problem while running faster across the full tested batch-size range and using substantially less GPU memory, and that a stochastic variant we propose outperforms AD on a stationary problem. We further show that for neural architectures with inter-sample dependencies (e.g. BatchNorm, self-attention) the standard PyTorch autograd idiom is silently incorrect; the correct per-sample alternative is computationally infeasible at PINN-relevant batch sizes, while FD provides a forward-only approximation that is empirically an order of magnitude closer to the true per-sample derivative.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。