arXiv:2602.14580cs.LGstat.ML2026-02被引 1

让约束型强化学习算法可复现,且性能不降。

Replicable Constrained Bandits

  • 用可复现的乐观原则设计算法,保证多次运行结果一致。
  • 在T轮中,算法的损失和约束违反与非可复现算法相当。
  • 首次实现无约束MAB问题的可复现性,对理论有独立价值。

算法可复现性被提出以应对机器学习实验可复现性的需求。可复现在线学习算法在相同环境下多次执行时,以高概率产生相同的决策序列。本文首次研究约束型多臂老虎机(Constrained MAB)中的可复现性问题。在T轮交互中,学习者需在最大化奖励的同时满足多个约束。主要成果是:可复现算法的遗憾(regret)和约束违反(constraint violation)与非可复现算法在T上的渐近界一致。关键步骤是设计首个针对无约束MAB的可复现UCB类算法,证明了基于‘面对不确定性保持乐观’原则的算法可实现可复现性,这一结果本身具有独立意义。

原文摘要 · Abstract (English)

Algorithmic \emph{replicability} has recently been introduced to address the need for reproducible experiments in machine learning. A \emph{replicable online learning} algorithm is one that takes the same sequence of decisions across different executions in the same environment, with high probability. We initiate the study of algorithmic replicability in \emph{constrained} MAB problems, where a learner interacts with an unknown stochastic environment for $T$ rounds, seeking not only to maximize reward but also to satisfy multiple constraints. Our main result is that replicability can be achieved in constrained MABs. Specifically, we design replicable algorithms whose regret and constraint violation match those of non-replicable ones in terms of $T$. As a key step toward these guarantees, we develop the first replicable UCB-like algorithm for \emph{unconstrained} MABs, showing that algorithms that employ the optimism in-the-face-of-uncertainty principle can be replicable, a result that we believe is of independent interest.

强化学习可复现性多臂老虎机

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。