arXiv:2607.07623cs.LGcs.NA2026-07

改进李维-马夸特法在非线性优化中的几何一致性,提升收敛速度与稳定性。

Higher-Order Geometric Updates for Levenberg-Marquardt Method via Riemann Normal Coordinates

  • 基于黎曼法坐标重构更新路径,实现任意阶次的几何修正。
  • 在曲谷和秩亏问题中显著提升收敛性,大尺度势能拟合提速34倍。
  • 适合高精度物理信息神经网络与复杂优化任务的科研人员使用。

非线性最小二乘优化在回归、物理信息神经网络等机器学习任务中至关重要。此类问题具有天然的几何意义:模型预测构成数据空间中的流形,而参数化方式引入的参数效应曲率成为非线性主因。这暴露了李维-马夸特(LM)方法的局限——其切空间步长在参数坐标中以直线更新。测地加速虽提供二阶修正,但仅在无穷小步长下精确消除参数效应曲率。本文提出黎曼法坐标下的李维-马夸特方法(RNC-LM),通过重构测地方程,将测地加速扩展至任意阶修正,并构建具有更高重参数化一致性的有限步更新。沿生成的RNC曲线进行线搜索控制行进距离,同时保持代价接近标准LM。该方法按阶次消除残差加速度的切向分量,在移动切空间中使实际目标下降更符合LM的线性模型预测。在经典非线性最小二乘基准测试中,RNC-LM在曲谷和秩亏问题中改善了收敛性与鲁棒性。在反应扩散型PINN失败模式基准上,相对L2误差降至1e-3量级并恢复物理解释解。在大规模机器学习势能面拟合任务中,相较标准LM实现34倍加速。

原文摘要 · Abstract (English)

Nonlinear least-squares optimization is central to regression, physics-informed neural networks, and other machine-learning tasks. Such problems have a natural geometric interpretation, model predictions form a manifold in data space, while the chosen parameterization can introduce parameter-effects curvature that becomes a dominant source of nonlinearity. This exposes a limitation of the Levenberg-Marquardt (LM) method, its tangent-space step is applied as a straight update in parameter coordinates. Geodesic acceleration gives a second-order correction, but its removal of parameter-effect curvature is exact only in the infinitesimal-step limit. We propose a Riemann-normal-coordinate Levenberg-Marquardt method (RNC-LM) to improve this consistency for finite optimization steps. By reformulating the geodesic equation, RNC-LM extends geodesic acceleration to arbitrary-order corrections and constructs finite-step updates with progressively higher reparameterization consistency. A line search along the resulting RNC curve controls the traveled distance while keeping the cost close to standard LM. The method eliminates the tangential component of residual acceleration order by order in a moving tangent frame, making the actual objective reduction more consistent with the linear model prediction of LM. On classical nonlinear least-squares benchmarks, RNC-LM improves convergence and robustness in curved valleys and rank-deficient problems. On a reaction-diffusion PINN failure-mode benchmark, it reduces the relative L2 error to the order of 1e-3 and recovers a physically meaningful solution. On a large-scale machine-learning potential-energy-surface fitting task, it achieves a 34-fold speedup over standard LM.

优化算法几何优化PINNLM方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。