用算子分解统一分析种群优化方法的收敛性。
Operator Calculus for Population-Based Optimization: A Mean-Field Convergence Theory
- 将优化过程拆解为突变、选择、重组三类算子作用于概率分布。
- 在稳定条件下,系统演化可由保持算子结构的偏微分方程描述。
- 通过模块化能量函数证明搜索误差指数衰减,适用于多类算法验证。
种群和分布优化方法(如进化策略、共识优化、协方差矩阵自适应及视为分布动力学的随机梯度法)广泛用于非凸或黑箱问题,但其收敛分析仍分散于各算法特有技术。本文引入一种算子微积分框架:在恰当状态空间下,通过必要时扩展记忆或策略变量,将一大类方法统一表示为突变、选择、重组三个基本算子对概率测度的复合作用。在明确的稳定性和正则性条件下,复合算子具有预生成算子,其连续极限为保持算子分裂结构的输运-反应-跳跃(TRJ)偏微分方程。基于此,建立模块化李雅普诺夫原理:若状态空间李雅普诺夫函数在全生成算子下耗散且控制相关搜索空间度量,则状态空间李雅普诺夫泛函与诱导搜索误差呈指数衰减。加性生成算子结构允许逐算子构造耗散估计,形成可认证复合平均场算法收敛性的工具包。
原文摘要 · Abstract (English)
Population-based and distributional optimization methods, from evolution strategies and consensus-based optimization to covariance-matrix adaptation and stochastic gradient methods viewed as distributional dynamics, are widely used for nonconvex or black-box problems, yet their convergence analyses remain fragmented across algorithm-specific techniques. We introduce an operator calculus in which a broad class of such methods, after choosing an appropriate state space and, where necessary, augmenting the state by memory or strategy variables, is described as a composition of three elementary operators (mutation, selection, and recombination) acting on probability measures. Under explicit stability and regularity conditions, the composite operator admits a pre-generator whose continuous-time limit is a transport-reaction-jump (TRJ) PDE that preserves the operator splitting. On this foundation we establish a modular Lyapunov principle. If a state-space Lyapunov function both dissipates under the full generator and controls the relevant search-space gauges, then the state-space Lyapunov functional and the induced search errors decay exponentially. The additive generator structure allows dissipation estimates to be assembled operator by operator, providing a toolkit for certifying convergence of composite mean-field algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。