arXiv:2506.13120cs.LG2025-06ICML被引 4

用神经算子加速微分方程优化,解决训练低效与不稳定问题。

Accelerating PDE-Constrained Optimization by the Derivative of Neural Operators

  • 针对优化目标设计训练数据与专用训练方法,提升神经算子效率。
  • 引入虚拟傅里叶层增强导数学习,保障梯度精度。
  • 结合数值求解器实现混合优化,显著提升收敛稳定性。

PDE约束优化(PDECO)问题可通过基于梯度的代理模型(如神经算子)大幅加速,相比传统数值求解器。然而该方法面临两大挑战:(1) 数据效率低:缺乏针对优化任务的高效采样与有效训练策略;(2) 不稳定性:神经算子预测与梯度不准确易导致优化失败。为此,我们提出新框架:(1) 优化导向训练:利用传统优化算法完整步骤的数据,并采用专有训练方法训练神经算子;(2) 增强导数学习:引入虚拟傅里叶层,提升神经算子内部导数学习能力,这对基于梯度的优化至关重要;(3) 混合优化:将神经算子与数值求解器结合,为优化过程提供鲁棒正则化。大量实验表明,所提模型能准确学习算子及其导数,且混合优化方法展现出稳健收敛性。

原文摘要 · Abstract (English)

PDE-Constrained Optimization (PDECO) problems can be accelerated significantly by employing gradient-based methods with surrogate models like neural operators compared to traditional numerical solvers. However, this approach faces two key challenges: (1) **Data inefficiency**: Lack of efficient data sampling and effective training for neural operators, particularly for optimization purpose. (2) **Instability**: High risk of optimization derailment due to inaccurate neural operator predictions and gradients. To address these challenges, we propose a novel framework: (1) **Optimization-oriented training**: we leverage data from full steps of traditional optimization algorithms and employ a specialized training method for neural operators. (2) **Enhanced derivative learning**: We introduce a *Virtual-Fourier* layer to enhance derivative learning within the neural operator, a crucial aspect for gradient-based optimization. (3) **Hybrid optimization**: We implement a hybrid approach that integrates neural operators with numerical solvers, providing robust regularization for the optimization process. Our extensive experimental results demonstrate the effectiveness of our model in accurately learning operators and their derivatives. Furthermore, our hybrid optimization approach exhibits robust convergence.

PDE优化神经算子梯度加速混合求解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。