提出一种降低方差的老虎机算法，追求稳定回报而非高风险高收益。
Softmax gradient policy for variance minimization and risk-averse multi armed bandits
- 用softmax策略建模，通过两次独立采样无偏估计方差目标
- 理论证明在自然条件下算法能收敛到方差最小的臂
- 适合关注稳定性、需平衡收益与风险的决策场景
多臂老虎机（MAB）算法在序列决策中扮演核心角色，已被广泛研究。传统方法通常寻找期望回报最高的臂，而本文聚焦风险感知场景，目标是选择方差最小的臂，以追求稳定性而非高但不确定的回报。采用softmax参数化策略，提出一种新算法来选择最小方差（或最低风险）的臂，并在自然条件下证明其收敛性。算法通过从当前臂分布中进行两次独立抽样，构建目标函数的无偏估计。数值实验展示了算法的实际表现，并提供了实现建议。该框架还可推广至一般风险感知问题，即在最大化平均回报与最小化方差之间权衡。
原文摘要 · Abstract (English)
Algorithms for the Multi-Armed Bandit (MAB) problem play a central role in sequential decision-making and have been extensively explored both theoretically and numerically. While most classical approaches aim to identify the arm with the highest expected reward, we focus on a risk-aware setting where the goal is to select the arm with the lowest variance, favoring stability over potentially high but uncertain returns. To model the decision process, we consider a softmax parameterization of the policy; we propose a new algorithm to select the minimal variance (or minimal risk) arm and prove its convergence under natural conditions. The algorithm constructs an unbiased estimate of the objective by using two independent draws from the current's arm distribution. We provide numerical experiments that illustrate the practical behavior of these algorithms and offer guidance on implementation choices. The setting also covers general risk-aware problems where there is a trade-off between maximizing the average reward and minimizing its variance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。