提出更合理的线性探索误差模型,让经典算法自动抗偏差并实现近最优后悔上界。
No-Regret Linear Bandits under Gap-Adjusted Misspecification
- 引入与子优度间隙成比例的误差模型,强调近优区域更重要
- 经典LinUCB在参数ρ随时间衰减时达到O(√T)后悔上界
- 新算法仅需log T批探索,适应性强且可部署,适合高维在线决策场景
本文研究在一种新的间隙调整型误设下的线性带宽问题,是对Liu等(2023)工作的扩展。当底层奖励函数非线性时,现有线性带宽方法通常依赖于统一的误设参数ε(衡量最优线性逼近的上确界范数误差),导致ε > 0时不可避免地出现线性后悔。我们提出更自然的误设模型:每个输入x处的逼近误差与该点的子优度间隙成正比,体现优化问题中近优区域应更受关注,次优区域可容忍更大误差的直觉。令人惊讶的是,经典LinUCB算法——专为可实现情形设计——对这种ρ-间隙调整误设具有天然鲁棒性,其中ρ以O(1/(d√log T))的速度衰减,仍能实现近乎最优的O(√T)后悔。我们进一步提出一种基于分阶段消除的新算法,其间隙调整误设参数ρ=O(1/√d),不随时间T变化,达到最优的O(√T)后悔,且部署高效,仅需log T批探索。此外,在存在常数子优度间隙时,其后悔可自适应降至O(log T)。技术上,证明依赖于新颖的自界论证,将误设导致的后悔部分绑定至自身,并结合新的归纳引理,保证每批次中由G-最优设计选择的有效动作的误设误差被限制在子优度间隙内。
原文摘要 · Abstract (English)
This work studies linear bandits under a new notion of gap-adjusted misspecification and is an extension of Liu et al. (2023). When the underlying reward function is not linear, existing linear bandits work usually relies on a uniform misspecification parameter $ε$ that measures the sup-norm error of the best linear approximation. This results in an unavoidable linear regret whenever $ε> 0$. We propose a more natural model of misspecification which only requires the approximation error at each input $x$ to be proportional to the suboptimality gap at $x$. It captures the intuition that, for optimization problems, near-optimal regions should matter more and we can tolerate larger approximation errors in suboptimal regions. Quite surprisingly, we show that the classical LinUCB algorithm -- designed for the realizable case -- is automatically robust against such $ρ$-gap-adjusted misspecification with parameter $ρ$ diminishing at $O(1/(d \sqrt{\log T}))$. It achieves a near-optimal $O(\sqrt{T})$ regret for problems that the best-known regret is almost linear in time horizon $T$. We further advance this frontier by presenting a novel phased elimination-based algorithm whose gap-adjusted misspecification parameter $ρ= O(1/\sqrt{d})$ does not scale with $T$. This algorithm attains optimal $O(\sqrt{T})$ regret and is deployment-efficient, requiring only $\log T$ batches of exploration. It also enjoys an adaptive $O(\log T)$ regret when a constant suboptimality gap exists. Technically, our proof relies on a novel self-bounding argument that bounds the part of the regret due to misspecification by the regret itself, and a new inductive lemma that limits the misspecification error within the suboptimality gap for all valid actions in each batch selected by G-optimal design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。