针对复杂演化问题的参数化模型，提出稳定高效的数值求解方法。
Regularized dynamical parametric approximation of stiff evolution problems
- 用正则化高斯-牛顿法近似求解参数演化中的非线性优化问题。
- 理论证明误差可控制，且在每步迭代中仅需少量计算。
- 适合处理神经网络、张量网络等过度参数化系统的动态模拟。
演化型深度神经网络已成为快速发展的研究领域。本文研究此类及其它非线性参数化形式 $ u(t) = Φ(θ(t)) $ 的数值积分方法，其中演化参数 $θ(t)$ 需被求解。重点解决刚性演化问题与不规则参数化结合带来的挑战，常见于神经网络、张量网络、动态高斯簇及过度参数化情形。本文提出并分析了隐式欧拉法和高阶隐式龙格-库塔法的正则化参数化版本，用于对演化偏微分方程和大规模刚性常微分方程系统进行参数时间积分。每个时间步通过少量正则化高斯-牛顿迭代近似求解病态非线性优化问题。通过将可计算的参数高斯-牛顿迭代与不可计算的非参数化时间积分牛顿迭代相联系，推导出所提参数积分器的误差界。理论结果得到设计精良的数值实验支持，验证了其关键性质。
原文摘要 · Abstract (English)
Evolutionary deep neural networks have emerged as a rapidly growing field of research. This paper studies numerical integrators for such and other classes of nonlinear parametrizations $ u(t) = Φ(θ(t)) $, where the evolving parameters $θ(t)$ are to be computed. The primary focus is on tackling the challenges posed by the combination of stiff evolution problems and irregular parametrizations, which typically arise with neural networks, tensor networks, flocks of evolving Gaussians, and in further cases of overparametrization. We propose and analyse regularized parametric versions of the implicit Euler method and higher-order implicit Runge--Kutta methods for the time integration of the parameters in nonlinear approximations to evolutionary partial differential equations and large systems of stiff ordinary differential equations. At each time step, an ill-conditioned nonlinear optimization problem is solved approximately with a few regularized Gauss--Newton iterations. Error bounds for the resulting parametric integrator are derived by relating the computationally accessible Gauss--Newton iteration for the parameters to the computationally inaccessible Newton iteration for the underlying non-parametric time integration scheme. The theoretical findings are supported by numerical experiments that are designed to show key properties of the proposed parametric integrators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。