提出新算法,显著降低在线优化中的约束违规,适用于安全关键场景。
Beyond $\tilde{O}(\sqrt{T})$ Constraint Violation for Online Convex Optimization with Adversarial Constraints
- 通过可调参数平衡损失与约束违规,实现更优的约束满足。
- 在高维情况下,约束违规可低至约 $T^{1-β}$,远优于传统 $\tilde{O}(\sqrt{T})$。
- 适用于需要严格遵守安全约束的实时决策系统,如自动驾驶、医疗控制。
研究对抗性约束下的在线凸优化(COCO)问题。每轮中,学习者从凸决策集选择动作,随后对手揭示凸代价函数和凸约束函数。目标是使累计损失和累计约束违规(CCV)在 $T$ 轮内最小化。现有最优策略的遗憾为 $O(\sqrt{T})$,CCV 为 $\tilde{O}(\sqrt{T})$。本文提出新策略,通过牺牲部分遗憾来显著降低 CCV。对任意有界凸代价与约束函数,构造策略实现 $\tilde{O}(\sqrt{dT}+ T^β)$ 遗憾与 $\tilde{O}(dT^{1-β})$ CCV,其中 $d$ 为决策集维度,$β∈[0,1]$ 可调。首先处理特殊情形“受限专家”问题(决策集为概率单纯形,代价与约束为线性),利用新型自适应小损失遗憾界,设计高效算法,达成 $O(\sqrt{T\ln N}+T^β)$ 遗憾与 $\tilde{O}(T^{1-β} \ln N)$ CCV,$N$ 为专家数。原问题通过覆盖论证转化为该情形。进一步,在 $M$-光滑假设下,提出高效一阶算法,实现 $O(\sqrt{MT}+T^β)$ 遗憾与 $\tilde{O}(MT^{1-β})$ CCV。
原文摘要 · Abstract (English)
We study Online Convex Optimization with adversarial constraints (COCO). At each round a learner selects an action from a convex decision set and then an adversary reveals a convex cost and a convex constraint function. The goal of the learner is to select a sequence of actions to minimize both regret and the cumulative constraint violation (CCV) over a horizon of length $T$. The best-known policy for this problem achieves $O(\sqrt{T})$ regret and $\tilde{O}(\sqrt{T})$ CCV. In this paper, we improve this by trading off regret to achieve substantially smaller CCV. This trade-off is especially important in safety-critical applications, where satisfying the safety constraints is non-negotiable. Specifically, for any bounded convex cost and constraint functions, we propose an online policy that achieves $\tilde{O}(\sqrt{dT}+ T^β)$ regret and $\tilde{O}(dT^{1-β})$ CCV, where $d$ is the dimension of the decision set and $β\in [0,1]$ is a tunable parameter. We begin with a special case, called the $\textsf{Constrained Expert}$ problem, where the decision set is a probability simplex and the cost and constraint functions are linear. Leveraging a new adaptive small-loss regret bound, we propose a computationally efficient policy for the $\textsf{Constrained Expert}$ problem, that attains $O(\sqrt{T\ln N}+T^β)$ regret and $\tilde{O}(T^{1-β} \ln N)$ CCV for $N$ number of experts. The original problem is then reduced to the $\textsf{Constrained Expert}$ problem via a covering argument. Finally, with an additional $M$-smoothness assumption, we propose a computationally efficient first-order policy attaining $O(\sqrt{MT}+T^β)$ regret and $\tilde{O}(MT^{1-β})$ CCV.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。