从统计视角梳理强化学习中多臂老虎机问题的理论与方法
Selective Reviews of Bandit Problems in AI via a Statistical View
- 基于集中不等式与极小极大后悔界分析决策模型
- 对比频率派与贝叶斯算法在探索与利用间的权衡
- 适合对强化学习理论感兴趣的研究生与研究人员
强化学习是人工智能中的重要研究领域，关注智能体通过与环境交互来学习决策。其中关键子领域包括随机多臂老虎机（MAB）和连续多臂老虎机（SCAB），用于建模不确定性下的序列决策问题。本文综述了带宽问题的基础模型与假设，探讨了非渐近理论工具如集中不等式和极小极大后悔界，并比较了频率派与贝叶斯算法在处理探索-利用权衡方面的表现。此外，重点分析了K臂上下文老虎机与SCAB的方法论及其后悔率分析，揭示了SCAB与函数数据分析之间的联系。最后，总结了该领域的最新进展与持续挑战。
原文摘要 · Abstract (English)
Reinforcement Learning (RL) is a widely researched area in artificial intelligence that focuses on teaching agents decision-making through interactions with their environment. A key subset includes stochastic multi-armed bandit (MAB) and continuum-armed bandit (SCAB) problems, which model sequential decision-making under uncertainty. This review outlines the foundational models and assumptions of bandit problems, explores non-asymptotic theoretical tools like concentration inequalities and minimax regret bounds, and compares frequentist and Bayesian algorithms for managing exploration-exploitation trade-offs. Additionally, we explore K-armed contextual bandits and SCAB, focusing on their methodologies and regret analyses. We also examine the connections between SCAB problems and functional data analysis. Finally, we highlight recent advances and ongoing challenges in the field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。