提出新算法实现在线凸优化的低交替遗憾,加速博弈均衡收敛。
Alternating Regret for Online Convex Optimization
- 用连续Hedge算法降低交替遗憾至$\tilde{\mathcal{O}}(d^{2/3}T^{1/3})$
- 在凸凹零和博弈中以$\tilde{\mathcal{O}}(d^{2/3}/T^{2/3})$速率找纳什均衡
- 对光滑损失问题设计新正则化算法,无维度依赖且更优
受双人博弈中交替学习动态启发,Cevher等(2024)证明任意$T$轮对抗性在线线性优化(OLO)问题可实现$o(\sqrt{T})$交替遗憾,但未解决一般在线凸优化(OCO)是否成立。本文正面回答该问题:连续Hedge算法对任意$d$维对抗性OCO问题实现$\tilde{\mathcal{O}}(d^{\frac{2}{3}}T^{\frac{1}{3}})$交替遗憾。此结果表明,对应的交替学习动态可于$\tilde{\mathcal{O}}(d^{\frac{2}{3}}/T^{\frac{2}{3}})$速率找到凸凹零和博弈的纳什均衡或凸双人一般和博弈的粗相关均衡。为提升时间复杂度与维度依赖,我们提出基于三阶平滑共轭正则项的FTRL算法,适用于光滑自协变损失函数(如线性或二次损失)。实例化后,当决策集为$\ell_2$球时,算法实现$\tilde{\mathcal{O}}(T^{\frac{2}{5}})$交替遗憾且无维度依赖;对二次损失,更优$\tilde{\mathcal{O}}(T^{\frac{1}{3}})$。我们还给出算法特定的交替遗憾下界,包括一个令人惊讶的$Ω(\sqrt{T})$下界——针对广泛使用的后悔匹配变体。
原文摘要 · Abstract (English)
Motivated by alternating learning dynamics in two-player games, a recent work by Cevher et al.(2024) shows that $o(\sqrt{T})$ alternating regret is possible for any $T$-round adversarial Online Linear Optimization (OLO) problem, and left as an open question whether the same is true for general Online Convex Optimization (OCO). We answer this question in the affirmative by showing that the continuous Hedge algorithm achieves $\tilde{\mathcal{O}}(d^{\frac{2}{3}}T^{\frac{1}{3}})$ alternating regret for any adversarial $d$-dimensional OCO problems. We show that this implies an alternating learning dynamic that finds a Nash equilibrium for any convex-concave zero-sum games or a coarse correlated equilibrium for any convex two-player general-sum games at a rate of $\tilde{\mathcal{O}}(d^{\frac{2}{3}}/T^{\frac{2}{3}})$. To further improve the time complexity and/or the dimension dependence, we propose another simple algorithm, Follow-the-Regularized-Leader with a regularizer whose convex conjugate is 3rd-order smooth, for OCO with smooth and self-concordant loss functions (such as linear or quadratic losses). We instantiate our algorithm with different regularizers and show that, for example, when the decision set is the $\ell_2$ ball, our algorithm achieves $\tilde{\mathcal{O}}(T^{\frac{2}{5}})$ alternating regret with no dimension dependence (and a better $\tilde{\mathcal{O}}(T^{\frac{1}{3}})$ bound for quadratic losses). We complement our results by showing some algorithm-specific alternating regret lower bounds, including a somewhat surprising $Ω(\sqrt{T})$ lower bound for a Regret Matching variant that is widely used in alternating learning dynamics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。