arXiv:2512.16200cs.LGcs.NE2025-12被引 2

首次给出基于排序的零阶优化算法的显式非渐近查询复杂度。

Explicit and Non-asymptotic Query Complexities of Rank-Based Zeroth-order Algorithms on Smooth Functions

  • 提出简单秩基零阶算法,不依赖梯度信息仅用函数值排序。
  • 在光滑凸函数下,查询复杂度为 $\widetilde{\mathcal O}(\frac{dL}{μ}\log\frac{dL}{μδ}\log\frac{1}{\varepsilon})$。
  • 理论揭示秩基启发式为何高效,适合无梯度优化研究者参考。

基于排序的零阶(ZO)优化——仅依赖函数评估值的排序信息——具有强噪声鲁棒性和单调变换不变性,是CMA-ES、自然进化策略及基于排序的遗传算法等成功算法的基础。尽管广泛应用,现有理论分析仅提供渐近结果,无法给出选择前$k$方向算法的显式收敛速率。本文首次分析一个简单秩基零阶算法,建立首个显式且非渐近的查询复杂度。对于$d$维光滑凸函数,若函数$L$-光滑且$μ$-强凸,算法以至少$1-δ$概率找到$\varepsilon$-次优解,查询复杂度为$\widetilde{\mathcal O}(\frac{dL}{μ}\log\frac{dL}{μδ}\log\frac{1}{\varepsilon})$;对光滑非凸目标,复杂度为$\mathcal O(\frac{dL}{\varepsilon}\log\frac{1}{\varepsilon})$。分析新颖,避开传统漂移与信息几何方法,揭示了秩基启发式为何能实现高效零阶优化。

原文摘要 · Abstract (English)

Rank-based zeroth-order (ZO) optimization -- which relies only on the ordering of function evaluations -- offers strong robustness to noise and monotone transformations, and underlies many successful algorithms such as CMA-ES, natural evolution strategies, and rank-based genetic algorithms. Despite its widespread use, the theoretical understanding of rank-based ZO methods remains limited: existing analyses provide only asymptotic insights and do not yield explicit convergence rates for algorithms selecting the top-$k$ directions. This work closes this gap by analyzing a simple rank-based ZO algorithm and establishing the first \emph{explicit}, and \emph{non-asymptotic} query complexities. For a $d$-dimension problem, if the function is $L$-smooth and $μ$-strongly convex, the algorithm achieves $\widetilde{\mathcal O}\!\left(\frac{dL}μ\log\!\frac{dL}{μδ}\log\!\frac{1}{\varepsilon}\right)$ to find an $\varepsilon$-suboptimal solution, and for smooth nonconvex objectives it reaches $\mathcal O\!\left(\frac{dL}{\varepsilon}\log\!\frac{1}{\varepsilon}\right)$. Notation $\cO(\cdot)$ hides constant terms and $\widetilde{\mathcal O}(\cdot)$ hides extra $\log\log\frac{1}{\varepsilon}$ term. These query complexities hold with a probability at least $1-δ$ with $0<δ<1$. The analysis in this paper is novel and avoids classical drift and information-geometric techniques. Our analysis offers new insight into why rank-based heuristics lead to efficient ZO optimization.

零阶优化理论分析随机优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。