解决带随机延迟反馈的连续动作强化学习问题，提升实际应用中的决策效率。
Lipschitz Bandits with Stochastic Delayed Feedback
- 设计延迟感知的分层探索算法，适应不同延迟场景
- 在有界延迟下实现接近无延迟的最优性能，额外代价与最大延迟相关
- 提出分阶段学习策略，适用于任意延迟分布，理论近最优
Lipschitz 网络带宽问题将随机带宽扩展到度量空间上的连续动作集，其中期望奖励函数满足 Lipschitz 条件。本文研究存在随机延迟反馈的 Lipschitz 带宽问题，即奖励并非立即观测，而是经过随机延迟后才可获取。我们考虑有界和无界随机延迟两种情形，并设计了相应的算法，分别实现次线性后悔率。对于有界延迟，提出一种延迟感知的聚焦算法，在不损失延迟无偏情形下的最优性能基础上，仅增加与最大延迟 τ_max 成比例的额外项。对于无界延迟，提出一种新型分阶段学习策略，通过精心调度的时间区间累积可靠反馈，并建立后悔率下界，证明所提方法在对数因子内近乎最优。最后，实验验证了算法在多种延迟场景下的高效性。
原文摘要 · Abstract (English)
The Lipschitz bandit problem extends stochastic bandits to a continuous action set defined over a metric space, where the expected reward function satisfies a Lipschitz condition. In this work, we introduce a new problem of Lipschitz bandit in the presence of stochastic delayed feedback, where the rewards are not observed immediately but after a random delay. We consider both bounded and unbounded stochastic delays, and design algorithms that attain sublinear regret guarantees in each setting. For bounded delays, we propose a delay-aware zooming algorithm that retains the optimal performance of the delay-free setting up to an additional term that scales with the maximal delay $τ_{\max}$. For unbounded delays, we propose a novel phased learning strategy that accumulates reliable feedback over carefully scheduled intervals, and establish a regret lower bound showing that our method is nearly optimal up to logarithmic factors. Finally, we present experimental results to demonstrate the efficiency of our algorithms under various delay scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。