提出两阶段训练框架,让神经控制器稳定性分析更准更快。
Two-Stage Learning of Stabilizing Neural Controllers via Zubov Sampling and Iterative Domain Expansion
- 用Zubov方法直接估算稳定域边界,减少训练保守性。
- 新采样策略和域更新机制使吸引域体积比基线大5到1.5万倍。
- 无需昂贵的SMT求解器,验证速度提升40到1万倍,适合高维系统。
基于神经网络的控制策略在实践中表现优异,但缺乏稳定性和吸引域估计的保障,主要因训练与验证算法难于稳定且可扩展。尽管已有工作取得显著进展,但框架仍存在较大保守性。本文提出一种新颖的两阶段训练框架,联合合成控制器与李雅普诺夫函数,用于连续时间系统。通过借鉴Zubov方法直接刻画吸引域边界,设计新型数据采样策略与域更新机制,显著降低训练保守性。不同于依赖SMT求解器验证李雅普诺夫条件的现有方法,我们扩展了先进的神经网络验证器α,β-CROWN,使其可自动传播动态系统的雅可比矩阵边界,并引入新验证方案避免耗时的二分搜索。在多个高维非线性系统上进行数值实验,结果表明,所提训练方法生成的吸引域体积比基线大5至1.5×10⁵倍;验证效率相比传统SMT求解器dReal提升40至10,000倍。代码已开源:https://github.com/Verified-Intelligence/Two-Stage_Neural_Controller_Training。
原文摘要 · Abstract (English)
Learning-based neural network (NN) control policies have shown impressive empirical performance. However, obtaining stability guarantees and estimates of the region of attraction of these learned neural controllers is challenging due to the lack of stable and scalable training and verification algorithms. Although previous works in this area have achieved great success, much conservatism remains in their frameworks. In this work, we propose a novel two-stage training framework to jointly synthesize a controller and a Lyapunov function for continuous-time systems. By leveraging a Zubov-inspired region of attraction characterization to directly estimate stability boundaries, we propose a novel training-data sampling strategy and a domain-updating mechanism that significantly reduces the conservatism in training. Moreover, unlike existing works on continuous-time systems that rely on an SMT solver to formally verify the Lyapunov condition, we extend state-of-the-art neural network verifier $α,\!β$-CROWN with the capability of performing automatic bound propagation through the Jacobian of dynamical systems and a novel verification scheme that avoids expensive bisection. To demonstrate the effectiveness of our approach, we conduct numerical experiments by synthesizing and verifying controllers on several challenging nonlinear systems across multiple dimensions. We show that our training can yield region of attractions with volume $5 - 1.5\cdot 10^{5}$ times larger compared to the baselines, and our verification on continuous systems can be up to $40-10{,}000$ times faster compared to the traditional SMT solver dReal. Our code is available at https://github.com/Verified-Intelligence/Two-Stage_Neural_Controller_Training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。