arXiv:2409.05898cs.LGcs.AI2024-09被引 3

用简单逻辑保障复杂系统持续学习的安全性,可应对真实环境中的未知风险。

Simplex-enabled Safe Continual Learning Machine

  • 基于物理模型的双架构设计:高性能学生与高可信教师协同
  • 实测证明在连续学习各阶段均能保证安全,且有效缩小仿真到现实的差距
  • 适合对安全性要求极高的自主系统,如机器人、自动驾驶

本文提出SeC-Learning Machine:一种基于Simplex逻辑(以简驭繁)和物理约束深度强化学习(Phy-DRL)的安全持续学习框架,专为安全关键型自主系统设计。该机器由三部分构成:高性能-学生(HP-Student),即预训练但未完全验证的Phy-DRL模型,在真实设备中持续优化动作策略;高可信-教师(HA-Teacher),基于简化任务与物理模型的可验证设计,负责安全保障与纠正错误学习;以及协调器,管理两者间的切换与交互。该架构可在任意持续学习阶段确保全生命周期安全,有效缓解Sim2Real差距,并具备容忍真实环境中未知未知的能力。在小车-摆杆系统与真实四足机器人上的实验表明,其性能显著优于现有先进安全DRL框架。

原文摘要 · Abstract (English)

This paper proposes the SeC-Learning Machine: Simplex-enabled safe continual learning for safety-critical autonomous systems. The SeC-learning machine is built on Simplex logic (that is, ``using simplicity to control complexity'') and physics-regulated deep reinforcement learning (Phy-DRL). The SeC-learning machine thus constitutes HP (high performance)-Student, HA (high assurance)-Teacher, and Coordinator. Specifically, the HP-Student is a pre-trained high-performance but not fully verified Phy-DRL, continuing to learn in a real plant to tune the action policy to be safe. In contrast, the HA-Teacher is a mission-reduced, physics-model-based, and verified design. As a complementary, HA-Teacher has two missions: backing up safety and correcting unsafe learning. The Coordinator triggers the interaction and the switch between HP-Student and HA-Teacher. Powered by the three interactive components, the SeC-learning machine can i) assure lifetime safety (i.e., safety guarantee in any continual-learning stage, regardless of HP-Student's success or convergence), ii) address the Sim2Real gap, and iii) learn to tolerate unknown unknowns in real plants. The experiments on a cart-pole system and a real quadruped robot demonstrate the distinguished features of the SeC-learning machine, compared with continual learning built on state-of-the-art safe DRL frameworks with approaches to addressing the Sim2Real gap.

持续学习安全强化学习物理模型自主系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。