针对动态变化的奖励环境，提出自适应方差的强化版UCB算法
From Theory to Practice with RAVEN-UCB: Addressing Non-Stationarity in Multi-Armed Bandits through Variance Adaptation
- 用方差调整探索策略，提升对变化环境的响应能力
- 理论证明其后悔率优于经典UCB1和UCB-V，达到最优量级
- 适合需要实时决策的动态系统，如物流调度、推荐系统
多臂赌博机（MAB）在奖励分布动态演变的非平稳环境中面临挑战。本文提出RAVEN-UCB，一种融合理论严谨性与实践高效性的新算法，通过方差感知适应实现更紧的后悔界：间隙依赖后悔率为$K σ_{ ext{max}}^2 \log T / Δ$，间隙独立后悔率为$\sqrt{K T \log T}$。该算法包含三项创新：(1) 在置信区间中使用$\sqrt{\hatσ_k^2 / (N_k + 1)}$驱动探索；(2) 采用$α_t = α_0 / \log(t + ε)$实现自适应控制；(3) 支持常数时间递推更新以提升效率。在合成数据及物流场景中，针对分布突变、周期性变化和短暂波动等非平稳模式的实验表明，RAVEN-UCB显著优于现有先进基线，验证了其理论与实际鲁棒性。
原文摘要 · Abstract (English)
The Multi-Armed Bandit (MAB) problem is challenging in non-stationary environments where reward distributions evolve dynamically. We introduce RAVEN-UCB, a novel algorithm that combines theoretical rigor with practical efficiency via variance-aware adaptation. It achieves tighter regret bounds than UCB1 and UCB-V, with gap-dependent regret of order $K σ_{\max}^2 \log T / Δ$ and gap-independent regret of order $\sqrt{K T \log T}$. RAVEN-UCB incorporates three innovations: (1) variance-driven exploration using $\sqrt{\hatσ_k^2 / (N_k + 1)}$ in confidence bounds, (2) adaptive control via $α_t = α_0 / \log(t + ε)$, and (3) constant-time recursive updates for efficiency. Experiments across non-stationary patterns - distributional changes, periodic shifts, and temporary fluctuations - in synthetic and logistics scenarios demonstrate its superiority over state-of-the-art baselines, confirming theoretical and practical robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。