L2正则化在Softmax MAB中消失时仍能收敛，提升算法性能。
Vanishing L2 regularization for the softmax Multi Armed Bandit
- 在软最大策略梯度中引入L2正则项，使优化更稳定。
- 理论证明当正则参数趋近零时算法依然收敛。
- 实验证明该设定在标准测试集上表现更优，适合强化学习初学者。
多臂赌博机（MAB）算法是强化学习的核心，常使用软最大映射定义最优策略，并成为下游算法（如REINFORCE）的基础。本文研究在软最大策略梯度中加入L2正则项的情形，即从均值奖励中减去二次项。以往基于凸性的分析无法处理正则参数趋近于零时的收敛性问题。本文首次提供理论收敛证明，并通过实验验证，在正则参数消失的极限下，该方法在标准基准测试中仍具数值优势。
原文摘要 · Abstract (English)
Multi Armed Bandit (MAB) algorithms are a cornerstone of reinforcement learning and have been studied both theoretically and numerically. One of the most commonly used implementation uses a softmax mapping to prescribe the optimal policy and served as the foundation for downstream algorithms, including REINFORCE. Distinct from vanilla approaches, we consider here the L2 regularized softmax policy gradient where a quadratic term is subtracted from the mean reward. Previous studies exploiting convexity failed to identify a suitable theoretical framework to analyze its convergence when the regularization parameter vanishes. We prove here theoretical convergence results and confirm empirically that this regime makes the L2 regularization numerically advantageous on standard benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。