通过分层辅助变量设计新模型，提升求解偏微分方程的深度学习效果。
Layer Separation Deep Learning Model with Auxiliary Variables for Partial Differential Equations
- 引入辅助变量分离网络层，将深层拆为多个浅层结构
- 新损失函数仅耦合相邻层，优化时可闭式求解多变量
- 理论保证与原模型一致，高维测试显示误差更低
本文提出一种新的优化框架——层分离（LySep）模型，以改进基于深度学习求解偏微分方程的方法。由于深度学习中损失函数高度非凸，现有优化算法常陷入次优局部极小值，或遭遇梯度爆炸/消失问题，导致性能不佳。为此，我们引入辅助变量，将神经网络各层的输出及其导数用辅助变量表示，从而将深层架构分解为一系列浅层架构。构建了含辅助变量的新损失函数，其中仅相邻两层变量相互耦合。相应地，设计了基于交替方向的算法，使多个变量可闭式最优更新。此外，我们提供了理论分析，证明了LySep模型与原始深度模型的一致性。高维数值实验验证了理论，并展示了LySep在降低损失和减少解误差方面的优势。
原文摘要 · Abstract (English)
In this paper, we propose a new optimization framework, the layer separation (LySep) model, to improve the deep learning-based methods in solving partial differential equations. Due to the highly non-convex nature of the loss function in deep learning, existing optimization algorithms often converge to suboptimal local minima or suffer from gradient explosion or vanishing, resulting in poor performance. To address these issues, we introduce auxiliary variables to separate the layers of deep neural networks. Specifically, the output and its derivatives of each layer are represented by auxiliary variables, effectively decomposing the deep architecture into a series of shallow architectures. New loss functions with auxiliary variables are established, in which only variables from two neighboring layers are coupled. Corresponding algorithms based on alternating directions are developed, where many variables can be updated optimally in closed forms. Moreover, we provide theoretical analyses demonstrating the consistency between the LySep model and the original deep model. High-dimensional numerical results validate our theory and demonstrate the advantages of LySep in minimizing loss and reducing solution error.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。