首个兼顾对抗、随机与安全基线的自适应在线学习算法
Learning Safely Without Knowing the World:COMPASS-Hedge
- 融合自适应伪损失缩放与分阶段激进策略,实现多环境协同优化
- 在对抗环境中达到最优后悔率,随机环境下实现间隙依赖最优
- 无需先验参数,适合复杂动态场景下的决策系统部署
在线学习算法常面临三重困境:在对抗与随机环境下平衡后悔率,同时提供相对于固定基准策略的安全性。现有方法通常只能在其中两项表现良好,且往往需依赖问题相关参数或牺牲最优率。本文提出COMPASS-Hedge,据我们所知是首个全信息场景下可实时运行的参数无关算法,能同时实现(忽略对数因子):i)对抗环境下的极小化最大后悔率;ii)随机环境下的实例最优、间隙依赖后悔率;iii)相对于指定基线策略的$ ilde{/mathcal{O}}(1)$后悔。该方法基于自适应伪后悔缩放、分阶段激进机制与比较器感知混合策略,首次在全信息设置中实现“三境最优”保障,证明基线安全性无需以最坏情况鲁棒性或随机效率为代价。
原文摘要 · Abstract (English)
Online learning algorithms often face a fundamental trilemma: balancing regret guarantees between adversarial and stochastic settings and providing baseline safety against a fixed comparator. While existing methods excel in one or two of these regimes, they typically fail to unify all three without sacrificing optimal rates or requiring oracle access to problem-dependent parameters. In this work, we bridge this gap by introducing COMPASS-Hedge. To the best of our knowledge, our algorithm is the first full-information anytime method to simultaneously achieve, up to logarithmic factors: i) minimax-optimal regret in adversarial environments; ii) instance-optimal, gap-dependent regret in stochastic environments; and iii) $\tilde{\mathcal{O}}(1)$ regret relative to a designated baseline policy. Crucially, COMPASS-Hedge is parameter-free and requires no prior knowledge of the environment's nature or the magnitude of the stochastic suboptimality gaps. Our approach hinges on a novel integration of adaptive pseudo-regret scaling and phase-based aggression, coupled with a comparator-aware mixing strategy. To the best of our knowledge, this provides the first "best-of-three-world" guarantee in the full-information setting, establishing that baseline safety does not have to come at the cost of worst-case robustness or stochastic efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。