让机器人像生物一样实时反应,自动避险。
Bresa: Bio-inspired Reflexive Safe Reinforcement Learning for Contact-Rich Robotic Tasks
- 分层设计:高层规划+低层安全反射,分别处理任务与风险。
- 实测中显著提升接触类任务安全性,动态环境适应性强。
- 适合需要实时避障的复杂机械臂操作场景。
在非结构化环境中执行接触丰富的机器人任务时,基于强化学习(RL)系统的安全性至关重要。现有先进安全强化学习方法虽通过安全探索或高层恢复机制降低风险,却常忽略底层执行安全——此时对潜在危险的即时反射响应尤为关键。同时,可变阻抗控制(VIC)虽能通过调节机器人的机械响应提升安全,但缺乏系统性参数自适应机制,难以动态调整刚度与阻尼。本文提出Bresa,一种受生物反射启发的分层安全强化学习方法。该方法将任务学习与安全学习解耦,引入一个运行频率高于任务求解器的安全评价网络,实时评估动作风险并进行低层干预。任务策略以较低频率运行,专注于高层规划;安全评价网络则在底层实时响应,确保瞬时安全。我们在多个任务上验证了Bresa,包括典型的接触密集型任务,结果表明其具备快速反射能力,并可在未预见动态环境中保持高安全性。实验显示,Bresa显著优于基线方法,构建了从高层规划到低层执行的安全桥梁。真实世界实验及补充材料见项目网站:https://jack-sherman01.github.io/Bresa。
原文摘要 · Abstract (English)
Ensuring safety in reinforcement learning (RL)-based robotic systems is a critical challenge, especially in contact-rich tasks within unstructured environments. While the state-of-the-art safe RL approaches mitigate risks through safe exploration or high-level recovery mechanisms, they often overlook low-level execution safety, where reflexive responses to potential hazards are crucial. Similarly, variable impedance control (VIC) enhances safety by adjusting the robot's mechanical response, yet lacks a systematic way to adapt parameters, such as stiffness and damping throughout the task. In this paper, we propose Bresa, a Bio-inspired Reflexive Hierarchical Safe RL method inspired by biological reflexes. Our method decouples task learning from safety learning, incorporating a safety critic network that evaluates action risks and operates at a higher frequency than the task solver. Unlike existing recovery-based methods, our safety critic functions at a low-level control layer, allowing real-time intervention when unsafe conditions arise. The task-solving RL policy, running at a lower frequency, focuses on high-level planning (decision-making), while the safety critic ensures instantaneous safety corrections. We validate Bresa on multiple tasks including a contact-rich robotic task, demonstrating its reflexive ability to enhance safety, and adaptability in unforeseen dynamic environments. Our results show that Bresa outperforms the baseline, providing a robust and reflexive safety mechanism that bridges the gap between high-level planning and low-level execution. Real-world experiments and supplementary material are available at project website https://jack-sherman01.github.io/Bresa.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。