arXiv:2606.14640cs.LGcs.DS2026-06中稿 · COLT '26

用少量噪声探针显著降低在线凸优化的最坏情况误差

Online Convex Optimization with Sublinear Noisy Probes

  • 引入统一探针模型,允许在子线性预算内进行成对比较
  • 理论证明探针可将后悔值降至 $ O\left(\frac{dT\ln T}{k|1-2δ|}\right) $
  • 适用于需低计算成本的实时决策场景,如在线推荐系统

研究定义在凸集 $K\subseteq \mathbb R^d$ 上的在线凸优化问题,每轮学习者选择 $x_t\in K$,随后观察损失函数 $f_t:K\to[0,1]$,目标是最小化相对于最优固定决策的后悔值。本文提出一个统一的探针模型,整合了专家设置中的子线性最优专家查询与在线凸优化中每轮可获得的成对反馈。在该框架下,学习者拥有最多 $k\le T$ 次成对探针机会;在探针轮次中,可查询两点并得知哪一点损失更小。主结果表明,即使探针数量为子线性且存在 $δ$-噪声,仍能严格改善全反馈在线凸优化下的最坏情况后悔率:$\text{Reg}_T \le O\left(\min\left\{\sqrt{dT\ln T},\; \frac{dT\ln T}{k|1-2δ|}\right\}\right)$,该界在 $T$、$k$ 与 $δ$ 上紧致(仅差对数因子)。特别地,当 $δ$ 接近 $\frac{1}{2}$(即响应趋近随机抛硬币)时,后悔率平滑退化。将相同技术应用于有限 $K$ 的 $d$ 个专家预测设置,所得率在所有参数上完全紧致。分析通过方差缩减效应量化探针收益,并结合基于二阶统计量的连续指数权重方法实现简化。

原文摘要 · Abstract (English)

We study Online Convex Optimization (OCO) over a convex set $K\subseteq \mathbb R^d$, where in each round $t$ the learner selects $x_t\in K$ and then observes a convex loss $f_t:K\to[0,1]$, with the goal of minimizing regret to the best fixed decision in hindsight. We introduce a unified probing model that generalizes two recent lines of work: sublinear best-expert queries in the experts setting, and pairwise (comparison-based) feedback available every round in OCO. In our framework, the learner has a budget of $k\le T$ pairwise probes; on a probed round it may query two points and learn which one has smaller loss. Our main result shows that even a sublinear and noisy probe budget can provably improve worst-case regret in the full feedback OCO regime. With $k$ $δ$-noisy pairwise probes, we obtain: $ \text{Reg}_T \le O\left(\min\left\{\sqrt{dT\ln T},\; \frac{dT\ln T}{k|1-2δ|}\right\}\right) $, which is tight (up to logarithmic factors in $T$) across $T$, $k$ and $δ$. Specifically regarding the noise parameter $δ\in [0,1]$, the regret guarantee smoothly degrades as the oracle response approaches a coin flip, i.e., $δ$ is close to $\frac{1}{2}$. When applying the same techniques to a finite $K$ for the prediction with $d$ experts setting, the resulting rates are instead completely tight in all parameters, including $d$. Our analysis gives a streamlined treatment of pairwise probing in OCO by quantifying the benefit of probing via a variance reduction effect, combined with a second-order (variance-based) analysis of Continuous Exponential Weights.

在线学习凸优化探针机制后悔界

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。