arXiv:2607.15003cs.AIcs.LG2026-07被引 1

用验证方法自动生成有安全保证的控制策略,兼顾性能与可靠性。

SMC-ES: Automated synthesis of formally verified control policies

  • 结合进化策略与统计模型检测,自动生成带证明证书的控制策略。
  • 在95%置信度下,违规概率不超过ε,满足性能与安全要求。
  • 适合需要严格安全验证的自动驾驶、工业控制等场景。

在安全关键环境中部署自主网络物理系统,需要闭环控制策略(即策略)不仅性能优异,还需具备形式化安全保障和鲁棒性。尽管基于学习的方法如强化学习能自动合成控制器,但通常缺乏安全部署所需的形式化保证。为此,我们提出一种新的基于仿真的方法,可自动合成具有形式化保证的策略,涵盖性能、安全性和鲁棒性规范。具体而言,在给定一组待验证属性、置信参数δ和允许失败概率ε的情况下,该方法确保合成的策略附带证书:以至少1−δ的置信度,违反这些属性的情形发生概率不超过ε。我们通过构建SMC-ES算法实现该方法,该算法将进化策略与基于统计模型检测的验证相结合。在Gymnasium和Safety Gymnasium测试平台上的连续控制任务中进行评估,结果表明,在可接受的计算成本增加下,SMC-ES在性能、安全性和鲁棒性方面均提供形式化保证,且表现媲美领先的无模型深度强化学习(DRL)和安全强化学习(Safe-DRL)基线。

原文摘要 · Abstract (English)

The deployment of autonomous cyber-physical systems in safety-critical environments requires closed-loop control strategies (i.e., policies) that are not only performant but also provably safe and robust. While learning-based methodologies such as Reinforcement Learning offer flexible and scalable approaches to automatically synthesize such controllers, they typically lack the formal guarantees necessary for safe deployment. To bridge this gap, we propose a novel simulation-based methodology to automatically synthesize policies with formal guarantees regarding performance, safety, and robustness specifications. Specifically, given a set of properties to verify, a confidence parameter $δ$ and an allowable failure probability $\varepsilon$, our method guarantees that the synthesized policy comes with a certificate: with confidence at least $1 - δ$, the probability of encountering a scenario where the given properties are violated is at most $\varepsilon$. We demonstrate the feasibility of our approach by developing SMC-ES, an algorithm that integrates Evolutionary Strategies with Statistical Model Checking-based verification. We evaluate SMC-ES on a suite of continuous control tasks using Gymnasium and Safety Gymnasium testbeds. Results show that, at the price of a sustainable increase in computational cost, our algorithm provides formal guarantees regarding performance, safety, and robustness specifications, while performing competitively against leading model-free Deep Reinforcement Learning (DRL) and Safe-DRL baselines.

控制策略形式验证强化学习安全控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。