arXiv:2607.12784cs.ROcs.LG2026-07中稿 · publication at the…

通过方向性约束提升安全强化学习的探索效率,减少无效约束干扰。

Directional Constraints for Efficient Exploration in Safe Reinforcement Learning

论文配图:Directional Constraints for Efficient Exploration in Safe Reinforcement Learning
图 1 · 摘自论文原文
  • 引入方向性约束,仅在靠近边界时激活安全限制。
  • 在多种机器人控制任务中降低约束违规率,同时保持任务性能。
  • 适合需高安全性与高效学习并重的复杂机器人场景。

强化学习已推动机器人研究革新,使复杂技能可在仿真中稳健学习。然而,在开放环境中的实际部署需要强安全保证,以防止危险或有害行为。安全强化学习方法通过施加安全约束来满足此需求。然而,约束下的学习常导致学习速度下降,并可能产生次优任务表现,因为相比无约束情况,代理需解决更复杂的约束优化问题。为应对这一挑战,本文提出对ATACOM框架的扩展,该框架是当前最先进的可靠安全层,可与现有强化学习算法集成,以实施来自系统先验知识或直接从数据中学习的约束。所提方法名为ATACOM方向性约束(ATACOM-DC),通过区分接近与远离约束边界的动作,仅在必要时激活约束保护,显著改善了安全与性能之间的权衡。我们在一系列具有挑战性的机器人控制任务仿真中评估了该方法,分析了约束违规成本和任务表现。代码及附加材料见 https://atacom-dc.robot-learning.net。

原文摘要 · Abstract (English)

Reinforcement Learning has revolutionized the landscape of robotic research, allowing robust learning of complex robotic skills in simulation. However, real-world deployment in open-ended environments requires strong safety guarantees to prevent dangerous or harmful behaviors. Safe Reinforcement Learning methods address this requirement by enforcing safety constraints. Nevertheless, learning under constraints often reduces learning speed and could lead to suboptimal task performance, as the agent must solve a more complex constrained optimization problem compared to unconstrained settings. To tackle this issue, in this work, we propose an extension of the ATACOM framework, a state-of-the-art reliable safety layer that can be integrated with existing Reinforcement Learning algorithms to enforce constraints derived from prior knowledge of the system or learned directly from data. Our proposed method, named ATACOM Directional Constraints (ATACOM-DC), significantly improves the safety-performance trade-off by introducing directional constraints that distinguish between actions approaching and moving away from constraint boundaries, activating constraint enforcement only when necessary. We evaluate our method across a range of challenging robotic control tasks in simulation, analyzing both constraint-violation costs and achieved task performance. Code and additional material at https://atacom-dc.robot-learning.net.

强化学习安全约束机器人控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。