为多目标优化的复杂度设下理论下界,揭示算法极限性能。
Complexity Bounds for Smooth Multiobjective Optimization
- 通过非退化嵌入法将单目标难题映射到多目标场景,保持下界不变。
- 非凸情形下需至少 $Ω(1/ε^2)$ 次迭代才能达到 $ε$-帕累托平稳点。
- 结果适用于各类一阶算法,对优化器设计具有指导意义。
我们研究了在光滑多目标优化中寻找 $\varepsilon$-帕累托平稳点的黑盒复杂度,以帕累托平稳性间隙 $\mathcal{G}(x)$(目标梯度的最佳凸组合范数)作为进展度量。分析基于一种非退化提升方法,将困难的单目标实例嵌入具有不同目标和非单点帕累托前沿的多目标问题,同时保持 $\mathcal{G}$ 的下界。主要结论包括:(i) 在 $μ$-强凸情形下,任意跨度一阶方法最坏情况下收敛速度不超过 $\exp(-Θ(T/\sqrtκ))$,需 $Θ(\sqrtκ\log(1/\varepsilon))$ 次迭代,与加速上界匹配;(ii) 在凸情形下,对盲式单步方法有 $Ω(1/T)$ 的最小迭代下界,对盲式跨度方法有 $Ω(1/T^2)$ 的通用末次迭代下界,进一步通过几何下界证明该界对一般自适应方法过松,实际为 $Ω(1/T)$;(iii) 在梯度 $L$-利普希茨的非凸情形下,$\mathcal{G}$ 的下界为 $Ω(\sqrt{L}/(T+1))$(阶数紧),意味着需 $Ω(1/\varepsilon^2)$ 次迭代才能达到 $\mathcal{G}(x)≤ε$,符合自然缩放。
原文摘要 · Abstract (English)
We study the oracle complexity of finding $\varepsilon$-Pareto stationary points in smooth multiobjective optimization with $m$ objectives. Progress is measured by the Pareto stationarity gap $\mathcal{G}(x)$, the norm of the best convex combination of objective gradients. Our analysis relies on a non-degenerate lifting that embeds hard single-objective instances into MOO instances with distinct objectives and non-singleton Pareto fronts while preserving lower bounds on $\mathcal{G}$. We establish: (i) in the $μ$-strongly convex case, any span first-order method has worst-case linear convergence no faster than $\exp(-Θ(T/\sqrtκ))$ after $T$ oracle calls, yielding $Θ(\sqrtκ\log(1/\varepsilon))$ iterations and matching accelerated upper bounds; (ii) in the convex case, an $Ω(1/T)$ min-iterate lower bound for oblivious one-step methods and a universal last-iterate lower bound $Ω(1/T^2)$ for oblivious span methods via polynomial-degree arguments, and we further show this latter bound is loose (for general adaptive methods) by importing geometric lower bounds to obtain an $Ω(1/T)$ min-iterate lower bound for general adaptive first-order methods; (iii) in the nonconvex case with $L$-Lipschitz gradients, an $Ω(\sqrt{L}/(T+1))$-type lower bound on $\mathcal{G}$ (tight in order), implying $Ω(1/\varepsilon^2)$ iterations to reach $\mathcal{G}(x)\le\varepsilon$ up to natural scaling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。