arXiv:2510.18074cs.LGcs.AI2025-10

让强化学习策略在不确定环境下保证成功概率,兼顾可靠性与效率。

R2L: Reliable Reinforcement Learning: Guaranteed Return & Reliable Policies in Reinforcement Learning

  • 用状态扩展法将可靠强化学习转化为标准问题,可直接套用现有算法。
  • 目标是最大化回报超过阈值的概率,而非仅优化平均收益。
  • 适合路由、资源分配等需保障成功率的高风险决策场景。

本文针对强化学习中可靠策略的设计问题,提出一种新范式:最大化累积回报超过预设阈值的概率,以满足实际应用中对性能保障的需求。该问题可通过状态增广方式转化为标准强化学习问题,从而无需设计全新算法即可使用如Q-learning或Dueling Double DQN等成熟方法。理论证明了两种形式的等价性,并在可靠路由任务中验证:目标不再是期望时间最小化,而是最大化在给定时间预算内抵达目的地的概率。实验表明,所提方法能有效平衡效率与可靠性,适用于随机性高且安全敏感的应用场景。

原文摘要 · Abstract (English)

In this work, we address the problem of determining reliable policies in reinforcement learning (RL), with a focus on optimization under uncertainty and the need for performance guarantees. While classical RL algorithms aim at maximizing the expected return, many real-world applications - such as routing, resource allocation, or sequential decision-making under risk - require strategies that ensure not only high average performance but also a guaranteed probability of success. To this end, we propose a novel formulation in which the objective is to maximize the probability that the cumulative return exceeds a prescribed threshold. We demonstrate that this reliable RL problem can be reformulated, via a state-augmented representation, into a standard RL problem, thereby allowing the use of existing RL and deep RL algorithms without the need for entirely new algorithmic frameworks. Theoretical results establish the equivalence of the two formulations and show that reliable strategies can be derived by appropriately adapting well-known methods such as Q-learning or Dueling Double DQN. To illustrate the practical relevance of the approach, we consider the problem of reliable routing, where the goal is not to minimize the expected travel time but rather to maximize the probability of reaching the destination within a given time budget. Numerical experiments confirm that the proposed formulation leads to policies that effectively balance efficiency and reliability, highlighting the potential of reliable RL for applications in stochastic and safety-critical environments.

强化学习可靠性决策保障状态增广

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。