用高斯过程预测风险，动态干预确保强化学习安全
Safe Reinforcement Learning via Recovery-based Shielding with Gaussian Process Dynamics Models
- 以高斯过程建模系统不确定性，实时预警安全风险
- 仅在必要时触发安全恢复策略，保障全程合规
- 支持自由探索且样本高效，适合高风险控制场景
强化学习在最优决策与控制中表现强大，但在安全关键应用中常缺乏可证明的安全保证。本文提出一种基于恢复的屏蔽框架，为未知且非线性的连续动力系统提供可证明的安全下界。该方法将备份策略（屏蔽器）与强化学习代理结合，利用高斯过程（GP）进行不确定性量化，预测潜在的安全约束违反，并仅在必要时动态恢复至安全轨迹。由‘受保护’代理积累的经验用于构建GP模型，通过内部基于模型的采样实现策略优化，支持无限制探索和高效学习，同时不牺牲安全性。实验表明，该方法在一系列连续控制环境中展现出优异性能和严格的安全合规性。
原文摘要 · Abstract (English)
Reinforcement learning (RL) is a powerful framework for optimal decision-making and control but often lacks provable guarantees for safety-critical applications. In this paper, we introduce a novel recovery-based shielding framework that enables safe RL with a provable safety lower bound for unknown and non-linear continuous dynamical systems. The proposed approach integrates a backup policy (shield) with the RL agent, leveraging Gaussian process (GP) based uncertainty quantification to predict potential violations of safety constraints, dynamically recovering to safe trajectories only when necessary. Experience gathered by the 'shielded' agent is used to construct the GP models, with policy optimization via internal model-based sampling - enabling unrestricted exploration and sample efficient learning, without compromising safety. Empirically our approach demonstrates strong performance and strict safety-compliance on a suite of continuous control environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。