arXiv:2501.15189cs.LGcs.RO2025-01

提出算法验证神经网络屏障函数的安全部分,确保系统不进入危险状态。

Extracting Forward Invariant Sets from Neural Network-Based Control Barrier Functions

  • 结合神经网络可达性分析与超平面排列枚举,找出安全状态区域。
  • 可为浅层神经网络生成严格证明的安全集,覆盖真实场景中的控制器。
  • 适合关注自主系统安全性的研究人员和工程师使用。

训练神经网络(NN)作为屏障函数(BF)是提升自主动态系统安全性的一种流行方法。尽管在实践中已取得显著成功,但这些方法通常无法在理论上保证生成的函数确实是有效的屏障函数,从而削弱了其作为安全证书的可信度。本文研究如何形式化地认证一个学习得到的神经网络作为自治系统中状态规避的屏障函数:即计算出一个状态空间中的区域,使得该候选神经网络在该区域内被严格证明为屏障函数。为此,我们提出一种可靠算法,能高效生成浅层神经网络的安全证书集。该算法融合两种新方法:首先利用神经网络可达性工具识别系统轨迹上神经网络输出不增加的状态子集;随后采用一种新型超平面排列枚举算法,求取神经网络零下水平集与上述状态集的交集。由此,算法可严格确定神经网络被认证为屏障函数的状态子集。我们在两个案例研究中验证了该算法对真实世界神经网络作为屏障函数的有效性,并通过可扩展性实验展示了算法的高效性。

原文摘要 · Abstract (English)

Training Neural Networks (NNs) to serve as Barrier Functions (BFs) is a popular way to improve the safety of autonomous dynamical systems. Despite significant practical success, these methods are not generally guaranteed to produce true BFs in a provable sense, which undermines their intended use as safety certificates. In this paper, we consider the problem of formally certifying a learned NN as a BF with respect to state avoidance for an autonomous system: viz. computing a region of the state space on which the candidate NN is provably a BF. In particular, we propose a sound algorithm that efficiently produces such a certificate set for a shallow NN. Our algorithm combines two novel approaches: it first uses NN reachability tools to identify a subset of states for which the output of the NN does not increase along system trajectories; then, it uses a novel enumeration algorithm for hyperplane arrangements to find the intersection of the NN's zero-sub-level set with the first set of states. In this way, our algorithm soundly finds a subset of states on which the NN is certified as a BF. We further demonstrate the effectiveness of our algorithm at certifying for real-world NNs as BFs in two case studies. We complemented these with scalability experiments that demonstrate the efficiency of our algorithm.

神经网络安全控制屏障函数形式化验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。