提出新方法提升神经网络验证精度与效率,助力自动驾驶安全部署。
$(\text{DNN})^2$: Doubly Non-Negative Relaxations for Deep Neural Networks
- 设计基于特征值最大化的新型乘子搜索机制,突破双重非负松弛瓶颈。
- 实验显示其边界比标准SDP更紧,常逼近精确解,且可验证全局最优性。
- 适合需要高可靠性的神经网络安全验证场景,如自动驾驶感知与控制模块。
现有针对ReLU神经网络验证的线性规划(LP)和半定规划(SDP)松弛因松弛间隙过大导致安全保证过于保守。完全正定规划(CPP)虽能消除该间隙,但求解为NP难。其最廉价的可处理松弛——双重非负规划(DNN)——保留了关键约束作为半定规划,但规模超出内点法在实际应用中的处理能力。尽管布勒-蒙特罗(BM)分解已被用于提升基于SDP的验证可扩展性,但尚无适用于更严格的DNN公式的相应成果。主要障碍在于DNN中额外的非负性约束导致最优性认证所需的对偶乘子不唯一,使标准认证方法失效。本文提出一种新颖的特征值最大化程序,在非唯一乘子空间中搜索有效证书,即全局最优性保证。实验表明,所提方法$(\text{DNN})^2$生成的界始终优于标准SDP方法,常接近精确解;当存在有效证书时,其认证程序可确认全局最优性。这些结果是实现紧致、可认证且计算可扩展的神经网络验证保障的关键一步,对于在安全关键的自主系统中部署神经网络控制器与感知模块至关重要。
原文摘要 · Abstract (English)
Existing linear program (LP) and semidefinite program (SDP) relaxations for rectified linear unit (ReLU) neural network (NN) verification yield overly-conservative safety guarantees due to significant relaxation gaps. While the completely positive program (CPP) formulation closes this gap, it is NP-hard to solve. Its cheapest tractable relaxation, the doubly non-negative program (DNN), retains critical constraints as an SDP, but one whose size exceeds the reach of interior-point methods at practical scale. While Burer-Monteiro (BM) factorization has been applied to make SDP-based verification scalable, no such result exists for the strictly tighter DNN formulation. A key obstacle is that additional non-negativity constraints in the DNN cause dual multipliers for optimality certification to be non-unique, making standard certification methods inapplicable. We propose a novel eigenvalue maximization procedure that searches the non-unique multiplier space for a valid certificate, i.e. a global optimality guarantee. Experiments demonstrate that our approach $(\text{DNN})^2$ produces bounds consistently tighter than the standard SDP method, often matching the exact solution, and that our certification procedure confirms global optimality when a valid certificate exists. These results are a key step toward providing tight, certifiable, and computationally scalable verification guarantees needed to deploy neural network controllers and perception modules in safety-critical autonomous systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。