提出新算法实现约束在线学习的最优后悔与约束违规控制。
A Geometric Approach to Constrained Online Learning
- 采用嵌套投影机制,通过几何路径收缩性分析控制累积违规。
- 强凸损失下达对数级后悔与违规,凸损失下平方根级性能最优。
- 适用于资源受限、需严格满足约束的实时决策场景。
研究对抗性时变约束下的在线凸优化问题。每轮学习者在观测损失与约束前做出决策,与事后最优固定策略比较。目标是在最小化后悔的同时控制累积约束违规(CCV)。现有算法在强凸损失下实现 $O(\log T)$ 后悔与 $O(\sqrt{T\log T})$ CCV,凸损失下为 $O(\sqrt{T})$ 后悔与 $O(\sqrt{T}\log T)$ CCV。本文提出 NP-OGD 算法,通过迭代嵌套投影实现:强凸损失下 $O(\log T)$ 后悔与 $O(\log T)$ CCV;凸损失下 $O(\sqrt{T})$ 后悔与 $O(\sqrt{T})$ CCV。分析基于几何运动界:将嵌套投影轨迹提升至高一维后,在非标准范数下路径自收缩,利用自收缩曲线有限长度定理控制总投影移动量。还通过分层球面覆盖证明互补下界:强凸损失下,任何多项式次线性后悔算法的 CCV 至少为 $Ω((\log T)^{(d-1)/(d+1)}/\log\log T)$;凸损失下,弱适应算法的下界为 $Ω(T^{(d-1)/(2(d+3))})$,NP-OGD 为 $Ω(T^{(d-1)/(2d)})$。对于约束专家情形(N 个专家),主动 Hedge 算法实现 $O(\sqrt{T\log N})$ 后悔与 $O(N)$ CCV,且在足够长的时间内达到最小最大 CCV 下界。
原文摘要 · Abstract (English)
We study constrained online convex optimization with adversarial time-varying constraints. At each round the learner acts before observing the loss and constraint, and is compared with the best fixed action satisfying all constraints in hindsight. The goal is to obtain minimax-optimal regret while controlling cumulative constraint violation (CCV). Prior algorithms achieved $O(\log T)$ regret with $O(\sqrt{T\log T})$ CCV for strongly convex losses, and $O(\sqrt{T})$ regret with $O(\sqrt{T}\log T)$ CCV for convex losses. We present NP-OGD, an iterated nested-projection algorithm. For strongly convex losses it achieves $O(\log T)$ regret and $O(\log T)$ CCV; for convex losses it achieves $O(\sqrt{T})$ regret and $O(\sqrt{T})$ CCV. The analysis relies on a geometric movement bound: after lifting the nested projected-gradient trajectory to one higher dimension, the lifted path is self-contracted under a nonstandard norm, so a finite-length theorem for self-contracted curves controls the total projection movement. We also prove complementary lower bounds using layered sphere packings. For strongly convex losses, any online algorithm with polynomially sublinear regret can incur CCV at least $Ω((\log T)^{(d-1)/(d+1)}/\log\log T)$. For convex losses, we prove CCV lower bounds $Ω(T^{(d-1)/(2(d+3))})$ for weakly adaptive algorithms and $Ω(T^{(d-1)/(2d)})$ for NP-OGD. Finally, for the constrained experts special case over $N$ experts, an active Hedge algorithm attains $O(\sqrt{T\log N})$ regret and $O(N)$ CCV, with a matching minimax CCV lower bound for sufficiently large horizons.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。