从离线数据学习神经安全屏障,无需模型或人工设计即可保障自主系统实时安全。
V-OCBF: Learning Safety Filters from Offline Data via Value-Guided Offline Control Barrier Functions
- 基于离线数据递归更新屏障函数,实现无模型的安全信息传播。
- 在多个场景中安全违规减少,任务性能仍保持良好。
- 适合需要离线训练、无在线交互的安全关键控制应用。
确保自主系统的安全性需要控制器在不依赖在线交互的情况下满足状态约束。现有安全离线强化学习方法通常仅强制执行软期望代价约束,难以保证严格的逐状态安全。相反,控制屏障函数(CBFs)提供了确保前向不变性的理论机制,但常依赖专家设计的屏障函数或系统动力学知识。本文提出价值引导的离线控制屏障函数(V-OCBF),一种完全从离线示范中学习神经CBF的框架。与以往方法不同,V-OCBF无需动力学模型;它采用递归有限差分屏障更新方式,实现屏障函数的无模型学习,并能随时间传播安全信息。此外,V-OCBF引入基于期望值的目标函数,避免对分布外动作查询屏障,并将更新限制在数据集支持的动作集合内。学习到的屏障函数通过二次规划(QP)公式实现实时安全控制合成。在多个案例研究中,V-OCBF显著减少了安全违规次数,同时保持优异的任务表现,验证了其在无需在线交互或手工设计屏障的情况下,可扩展用于安全关键控制器的离线合成。
原文摘要 · Abstract (English)
Ensuring safety in autonomous systems requires controllers that aim to satisfy state-wise constraints without relying on online interaction.While existing Safe Offline RL methods typically enforce soft expected-cost constraints, they struggle to ensure strict state-wise safety. Conversely, Control Barrier Functions (CBFs) offer a principled mechanism to enforce forward invariance, but often rely on expert-designed barrier functions or knowledge of the system dynamics. We introduce Value-Guided Offline Control Barrier Functions (V-OCBF), a framework that learns a neural CBF entirely from offline demonstrations. Unlike prior approaches, V-OCBF does not assume access to the dynamics model; instead, it derives a recursive finite-difference barrier update, enabling model-free learning of a barrier that propagates safety information over time. Moreover, V-OCBF incorporates an expectile-based objective that avoids querying the barrier on out-of-distribution actions and restricts updates to the dataset-supported action set. The learned barrier is then used with a Quadratic Program (QP) formulation to synthesize real-time safe control. Across multiple case studies, V-OCBF yields substantially fewer safety violations than baseline methods while maintaining strong task performance, highlighting its scalability for offline synthesis of safety-critical controllers without online interaction or hand-engineered barriers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。