用噪声方差增强不确定性，提升神经上下文老虎机的探索效率
Variance-Aware Linear UCB with Deep Representation for Neural Contextual Bandits
- 引入奖励噪声方差上界σ²ₜ改进置信区间，更精准衡量不确定性
- 理论证明算法在神经上下文老虎机中达到更优的后悔上界
- 实测在多个数据集上比现有方法更低后悔值且校准更优
通过利用深度神经网络的表示能力，神经上置信界（UCB）算法已在上下文老虎机问题中取得成功。为更好平衡探索与利用，本文提出神经-σ²-线性UCB，一种基于σ²ₜ（第t轮奖励噪声方差的上界）的方差感知算法，以提升UCB的不确定性量化质量，从而改善后悔性能。我们提供了一个使用理想方差上界的理论版本，以及一个具有新颖方差上界估计的实用版本。理论上，我们对两个版本均进行了严格的后悔分析，证明了其在神经上下文老虎机设置下优于其他神经UCB算法。实验表明，该实用方法计算效率相近，但在合成、UCI、MNIST和CIFAR-10等多个标准数据集上，表现更佳的校准能力和更低的后悔值。
原文摘要 · Abstract (English)
By leveraging the representation power of deep neural networks, neural upper confidence bound (UCB) algorithms have shown success in contextual bandits. To further balance the exploration and exploitation, we propose Neural-$σ^2$-LinearUCB, a variance-aware algorithm that utilizes $σ^2_t$, i.e., an upper bound of the reward noise variance at round $t$, to enhance the uncertainty quantification quality of the UCB, resulting in a regret performance improvement. We provide an oracle version for our algorithm characterized by an oracle variance upper bound $σ^2_t$ and a practical version with a novel estimation for this variance bound. Theoretically, we provide rigorous regret analysis for both versions and prove that our oracle algorithm achieves a better regret guarantee than other neural-UCB algorithms in the neural contextual bandits setting. Empirically, our practical method enjoys a similar computational efficiency, while outperforming state-of-the-art techniques by having a better calibration and lower regret across multiple standard settings, including on the synthetic, UCI, MNIST, and CIFAR-10 datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。