改进DeepONet结构,用Transformer思想提升求解微分方程的精度与效率。
Efficient Transformer-Inspired Variants of Physics-Informed Deep Operator Networks
- 引入双向交叉条件机制,让分支与主干网络动态交互。
- 在4个方程上均达到或超越改进版DeepONet的精度,训练更快。
- 适配不同物理特性,适合需要高效高精度的科学计算场景。
算子学习已成为加速偏微分方程(PDE)求解的有力工具。深度算子网络(DeepONet)是该领域的开创性框架:原始版本以简洁高效著称,而改进版虽精度更高但训练耗时增加。本文提出一系列受Transformer启发的DeepONet变体,通过在分支网络注入查询点信息、在主干网络注入输入函数信息,实现分支与主干间的双向交叉条件,动态建模依赖关系,同时保持原始DeepONet的简洁性和非侵入性。在四个典型PDE基准——对流方程、扩散-反应方程、Burgers方程和Korteweg-de Vries方程上的实验表明,每类方程均有对应变体在精度上匹配或超越改进版DeepONet,且训练效率显著提升。最优变体的表现自然契合各方程的物理特性,说明交叉条件的有效性取决于方程本身的物理特征。通过威尔科xon双单侧检验、Glass's Delta和斯皮尔曼等级相关等严格统计分析验证了方法的鲁棒性。
原文摘要 · Abstract (English)
Operator learning has emerged as a promising tool for accelerating the solution of partial differential equations (PDEs). The Deep Operator Networks (DeepONets) represent a pioneering framework in this area: the "vanilla" DeepONet is valued for its simplicity and efficiency, while the modified DeepONet achieves higher accuracy at the cost of increased training time. In this work, we propose a series of Transformer-inspired DeepONet variants that introduce bidirectional cross-conditioning between the branch and trunk networks in DeepONet. Query-point information is injected into the branch network and input-function information into the trunk network, enabling dynamic dependencies while preserving the simplicity and efficiency of the "vanilla" DeepONet in a non-intrusive manner. Experiments on four PDE benchmarks -- advection, diffusion-reaction, Burgers', and Korteweg-de Vries equations -- show that for each case, there exists a variant that matches or surpasses the accuracy of the modified DeepONet while offering improved training efficiency. Moreover, the best-performing variant for each equation aligns naturally with the equation's underlying characteristics, suggesting that the effectiveness of cross-conditioning depends on the characteristics of the equation and its underlying physics. To ensure robustness, we validate the effectiveness of our variants through a range of rigorous statistical analyses, among them the Wilcoxon Two One-Sided Test, Glass's Delta, and Spearman's rank correlation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。