arXiv:2606.17523math.OCcs.LG2026-06中稿 · PPSN 2026

证明了连续空间中IGO算法的收敛性,为进化策略提供理论支持

Beyond IGO-Flow: Toward Convergence Analysis of IGO in Continuous Spaces

论文配图:Beyond IGO-Flow: Toward Convergence Analysis of IGO in Continuous Spaces
图 1 · 摘自论文原文
  • 在高斯分布上用自然梯度分析离散时间IGO,允许全协方差自适应
  • 协方差矩阵收敛至零,均值向全局最优收敛,条件是协方差条件数受限
  • 结果连接理论与CMA-ES等实际优化方法,适合优化算法研究者

信息几何优化(IGO)通过将搜索分布的调整视为自然梯度更新,为黑箱优化提供了统一框架。尽管概念重要,现有收敛理论多限于连续时间理想化情形(如IGO流),而非具有非无穷小学习率的离散时间更新。本文研究连续空间中的离散时间IGO,将其表述为指数族期望参数坐标下的自然梯度更新。特别地,我们在强凸二次目标函数上的多元高斯族上分析IGO,涵盖全协方差自适应、固定正学习率及基于分位数的权重设置。结果表明,协方差矩阵收敛至零矩阵;在适当缩放的协方差矩阵条件数于足够频繁迭代中受控时,均值向量收敛至全局最优。这些成果推进了IGO的收敛理论,弥合了其数学理论与如CMA-ES等实际协方差自适应搜索方法之间的差距。

原文摘要 · Abstract (English)

Information-Geometric Optimization (IGO) provides a unified framework for black-box optimization by interpreting the adaptation of a search distribution as a natural gradient update. Despite its conceptual importance, the convergence theory of IGO remains limited: most existing results concern continuous-time idealizations such as the IGO flow, rather than discrete-time updates with non-infinitesimal learning rates. In this paper, we study discrete-time IGO in continuous spaces, formulated as natural gradient updates in the expectation-parameter coordinates of an exponential family. In particular, we analyze IGO over the multivariate Gaussian family on strongly convex quadratic objective functions. Our analysis covers a setting that simultaneously incorporates full covariance adaptation, a fixed positive learning rate, and quantile-based weights. In this setting, we prove that the covariance matrix converges to the zero matrix. We further show that the mean vector converges to the global optimum, provided that the condition number of the appropriately scaled covariance matrix is bounded at sufficiently frequent iterations. These results advance the convergence theory of IGO and help bridge the gap between the mathematical theory of IGO and practical covariance-adaptive search methods such as CMA-ES.

优化算法收敛分析自然梯度CMA-ES

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。