无需梯度的优化方法，用残差差分构建轻量代理模型提升非线性反问题求解效率。
Residual subspace evolution strategies for nonlinear inverse problems
- 通过高斯探测残差变化方向，用最小二乘重构更新方向，避免计算雅可比矩阵。
- 每次迭代仅需k+1次残差评估，复杂度O(k³)，k远小于参数维度，高效稳定。
- 在不平滑或协方差失效场景下表现优于xNES、Adam等方法，适合难处理反问题。
非线性反问题广泛存在于工程与科学领域，但噪声大、不可微或代价高的残差评估常使基于雅可比的求解器失效。现有无梯度方法要么要求平滑性，要么需大规模种群稳定协方差估计，或在梯度信息消失的平坦区域停滞。本文提出残差子空间演化策略（RSES），在当前迭代点周围施加高斯探测，记录残差沿这些方向的变化，并通过最小二乘求解重构最优更新方向。该方法仅依赖残差构建代理模型，无需构造雅可比矩阵或经验协方差，每轮迭代仅需$ k+1 $次残差评估，线性代数开销为$ O(k^3) $，其中$ k $远小于参数维度。在校准、回归和去卷积任务上的基准测试表明，RSES在确定性和随机设置下均一致降低误差，匹配或超越xNES、NEWUOA、Adam及集合卡尔曼反演，在评估预算相同条件下表现更优。当平滑性或协方差假设失效时优势尤为明显，说明轻量级残差差分代理模型能在重工具失效处仍可靠引导下降。
原文摘要 · Abstract (English)
Nonlinear inverse problems pervade engineering and science, yet noisy, non-differentiable, or expensive residual evaluations routinely defeat Jacobian-based solvers. Derivative-free alternatives either demand smoothness, require large populations to stabilise covariance estimates, or stall on flat regions where gradient information fades. This paper introduces residual subspace evolution strategies (RSES), a derivative-free solver that draws Gaussian probes around the current iterate, records how residuals change along those directions, and recombines the probes through a least-squares solve to produce an optimal update. The method builds a residual-only surrogate without forming Jacobians or empirical covariances, and each iteration costs just $k+1$ residual evaluations with $O(k^3)$ linear algebra overhead, where $k$ remains far smaller than the parameter dimension. Benchmarks on calibration, regression, and deconvolution tasks show that RSES reduces misfit consistently across deterministic and stochastic settings, matching or exceeding xNES, NEWUOA, Adam, and ensemble Kalman inversion under matched evaluation budgets. The gains are most pronounced when smoothness or covariance assumptions break, suggesting that lightweight residual-difference surrogates can reliably guide descent where heavier machinery struggles.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。