无需强假设,首次实现对数收益博弈的最优后悔率
Logistic Bandits with $\tilde{O}(\sqrt{dT})$ Regret without Context Diversity Assumptions

- 通过分离样本构建初始估计与牛顿修正,避免依赖上下文多样性
- 达到$ ilde{O}( oot dT)$后悔率,且不依赖维度$d$的显式依赖
- 适合上下文集中于低维子空间的场景,理论与实验均验证优势
我们研究$K$臂对数线性带宽问题,每轮观测$K$个动作对应的特征向量。现有方法实现$ ilde{ m O}( oot dT)$最优后悔率需依赖上下文多样性假设,如上下文协方差矩阵最小特征值严格为正,这限制了上下文分布,排除其集中在低维子空间的情况。本文提出SupSplitLog,据我们所知是首个无需任何上下文多样性假设即达到$ ilde{ m O}( oot dT)$后悔率的算法。核心思想是将采样数据分为两组:一组用于计算初始估计,另一组用于执行牛顿型一步校正。分割规则精心设计以平衡初始估计与校正的精度需求。此外,该算法在后悔上界中对维度$d$的依赖更优。还可简单扩展为基于数据依赖复杂度测度的后悔界,避免对$d$的直接依赖,适用于上下文集中于低维子空间的情形。实验结果数值验证了算法优越性,支持理论结论。
原文摘要 · Abstract (English)
We study the $K$-armed logistic bandit problem, where at each round, the agent observes $K$ feature vectors associated with $K$ actions. Existing approaches that achieve a rate-optimal $\tilde{\mathcal{O}}(\sqrt{dT})$ regret bound rely heavily on context diversity assumptions, such as strict positivity of the minimum eigenvalue of a context covariance matrix. These assumptions, however, impose strong restrictions on the context process, as they rule out the situation where the context vectors are concentrated in a low-dimensional subspace. In this paper, we propose SupSplitLog, which, to the best of our knowledge, is the first algorithm for logistic bandits that achieves $\tilde{\mathcal{O}}(\sqrt{dT})$ regret without any context diversity assumption. The key idea is to split the collected samples into two disjoint subsets when constructing estimators; one is used to compute an initial-point estimator, while the other is used to apply a Newton-type one-step correction procedure. The splitting rule is carefully designed to balance the accuracy requirements of the initial-point estimator and the one-step correction procedure. Moreover, SupSplitLog strictly improves on the existing algorithms in terms of the dependence on dimension $d$ in the regret upper bound. Furthermore, SupSplitLog can be adapted simply to deduce a regret bound that grows with a data-dependent complexity measure, avoiding a direct dependence on $d$, which is favorable when the context vectors are concentrated in a low-dimensional subspace. We also provide experimental results that demonstrate numerically the superiority of our algorithm, validating the theoretical results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。