arXiv:2605.01752cs.LG2026-05

解决延迟与干扰下的线性对决强化学习,性能不随延迟变化。

Robust Linear Dueling Bandits with Post-serving Context under Unknown Delays and Adversarial Corruptions

  • 用预反馈信息预测后反馈上下文,结合自适应加权抑制干扰和延迟影响。
  • 理论证明算法在未知延迟下仍保持近最优后悔上界,依赖维度、时间、干扰总量和延迟复杂度。
  • 适合研究鲁棒强化学习、在线决策系统或受干扰环境中的推荐系统开发者。

我们研究在存在后反馈上下文、反馈延迟和对抗性干扰的动态环境中线性对决强化学习问题。反馈受未知随机或对抗性延迟影响,并受限于累计干扰预算 $\\(mathcal{C}$$$。为此,我们提出 e RCDP-UCB 算法,利用学习的近似器从预反馈信息中预测后反馈上下文,并采用自适应加权策略对特征向量进行截断,以同时缓解干扰和延迟的影响。在标准正则条件下及参数化后反馈映射假设下,我们严格证明该算法具有延迟无关性,其后悔上界为 $\\(\widetilde{\mathcal{O}}(d(\sqrt{T} + \mathcal{C} + \mathcal{D}))$$,其中 $d$ 为特征维度,$\\(\mathcal{D}$$$ 表示延迟复杂度。关键发现是干扰与延迟的代价呈加法关系,避免了以往工作常见的乘法退化。我们进一步建立了下界,表明在无后反馈上下文时,上界仅差 $\\(\sqrt{d}$$$ 因子。代码已开源。

原文摘要 · Abstract (English)

We study linear dueling bandits in volatile environments characterized by the simultaneous presence of post-serving contexts, delayed feedback, and adversarial corruption. Feedback is subject to unknown stochastic or adversarial delays and a cumulative corruption budget $\mathcal{C}$. To address these challenges, we propose e RCDP-UCB, which integrates a learned approximator that predicts post-serving contexts from pre-serving information. It further employs an adaptive weighting strategy that clips feature vectors to mitigate the impact of corrupted and delayed observations simultaneously. Under standard regularity conditions and a parametric post-serving mapping, we rigorously establish that our algorithm is delay-regime-agnostic, achieving a regret upper bound of $\widetilde{\mathcal{O}}(d(\sqrt{T} + \mathcal{C} + \mathcal{D}))$, where $d$ is the total feature dimension and $\mathcal{D}$ encapsulates the delay complexity. Crucially, our analysis reveals an additive cost structure between corruption and delay, avoiding the multiplicative degradation typical of prior works. We further establish lower bounds that nearly match our upper bounds up to a $\sqrt{d}$ factor for adversarial delays in the absence of post-serving contexts. Code is available at https://github.com/youngmin0oh/rcdp-public.

强化学习对抗性干扰延迟反馈在线学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。