EQO算法用简单奖励机制实现最优探索，理论与实践兼优。
Minimax Optimal Reinforcement Learning with Quasi-Optimism
- 通过反向访问次数加奖励，实现无需方差估计的高效探索。
- 在最弱假设下达到最优后悔值，收敛速度最快。
- 适合追求理论严谨性与计算效率的强化学习研究者。
为实现兼具实用性与理论最优性的强化学习算法，我们提出EQO（基于准乐观性的探索）。不同于现有最小最大最优方法，EQO不依赖经验方差，采用与状态-动作访问次数倒数成比例的简单奖励项。核心在于准乐观性概念：估值无需完全乐观，仍可实现高效探索。该算法在最温和假设下实现了表格型强化学习的最尖锐已知后悔界，证明了快速收敛可通过实用且计算高效的策略达成。实验表明，EQO在后悔值和计算效率上均持续优于现有算法，兼具理论严谨性与实际有效性。
原文摘要 · Abstract (English)
In our quest for a reinforcement learning (RL) algorithm that is both practical and provably optimal, we introduce EQO (Exploration via Quasi-Optimism). Unlike existing minimax optimal approaches, EQO avoids reliance on empirical variances and employs a simple bonus term proportional to the inverse of the state-action visit count. Central to EQO is the concept of quasi-optimism, where estimated values need not be fully optimistic, allowing for a simpler yet effective exploration strategy. The algorithm achieves the sharpest known regret bound for tabular RL under the mildest assumptions, proving that fast convergence can be attained with a practical and computationally efficient approach. Empirical evaluations demonstrate that EQO consistently outperforms existing algorithms in both regret performance and computational efficiency, providing the best of both theoretical soundness and practical effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。