通过采样优化线性松弛,提升神经网络验证的精度与效率。
Probabilistically Tightened Linear Relaxation-based Perturbation Analysis for Neural Network Verification
- 结合采样方法优化线性松弛的中间可达集
- 验证精度提升最多3.31倍,计算成本低
- 适合处理先进方法失效的复杂验证任务
我们提出一种新框架PT-LiRPA,将LiRPA方法中的过度近似技术与基于采样的方法结合,以计算更紧致的中间可达集。实验表明,在几乎无额外计算开销下,该方法显著收紧了神经网络输出的上下界线性约束,降低了形式化验证工具的计算成本,同时提供验证可靠性的概率保障。在标准验证基准(包括国际神经网络验证竞赛)上的大量实验显示,基于PT-LiRPA的验证器将模型可认证的扰动容忍范围(ε的下界)相比已有方法最高提升3.31倍和2.26倍。更重要的是,该概率方法在现有先进验证方法失败的挑战性竞赛题目上仍能以至少99%的信心给出答案。
原文摘要 · Abstract (English)
We present $\textbf{P}$robabilistically $\textbf{T}$ightened $\textbf{Li}$near $\textbf{R}$elaxation-based $\textbf{P}$erturbation $\textbf{A}$nalysis ($\texttt{PT-LiRPA}$), a novel framework that combines over-approximation techniques from LiRPA-based approaches with a sampling-based method to compute tight intermediate reachable sets. In detail, we show that with negligible computational overhead, $\texttt{PT-LiRPA}$ exploiting the estimated reachable sets, significantly tightens the lower and upper linear bounds of a neural network's output, reducing the computational cost of formal verification tools while providing probabilistic guarantees on verification soundness. Extensive experiments on standard formal verification benchmarks, including the International Verification of Neural Networks Competition, show that our $\texttt{PT-LiRPA}$-based verifier improves robustness certificates, i.e., the certified lower bound of $\varepsilon$ perturbation tolerated by the models, by up to 3.31X and 2.26X compared to related work. Importantly, our probabilistic approach results in a valuable solution for challenging competition entries where state-of-the-art formal verification methods fail, allowing us to provide answers with high confidence (i.e., at least 99%).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。