提出更精确且可扩展的深度神经网络利普希茨界方法
Improved Scalable Lipschitz Bounds for Deep Neural Networks
- 基于更通用的参数化构造闭式解，避开传统求解器
- 实验表明相比ECLipsE-Fast有更优的界估计精度
- 适合关注模型鲁棒性与大规模网络分析的研究者
计算深度神经网络的紧致利普希茨界对分析其鲁棒性和稳定性至关重要，但现有方法要么估计过保守，要么依赖半定规划（SDP）形式（即LipSDP条件），面临可扩展性问题。本文在当前最优的无SDP方法ECLipsE-Fast基础上，推导出一类新的可扩展利普希茨界，可通过组合超越ECLipsE-Fast。具体地，利用LipSDP可行点的更一般参数化，导出多种闭式利普希茨界，避免使用SDP求解器。此外，我们证明该方法包含ECLipsE-Fast作为特例，并生成更大类别的可扩展利普希茨界。实证研究显示，我们的界优于ECLipsE-Fast，进一步提升了大规模神经网络利普希茨估计的可扩展性与精度。
原文摘要 · Abstract (English)
Computing tight Lipschitz bounds for deep neural networks is crucial for analyzing their robustness and stability, but existing approaches either produce relatively conservative estimates or rely on semidefinite programming (SDP) formulations (namely the LipSDP condition) that face scalability issues. Building upon ECLipsE-Fast, the state-of-the-art Lipschitz bound method that avoids SDP formulations, we derive a new family of improved scalable Lipschitz bounds that can be combined to outperform ECLipsE-Fast. Specifically, we leverage more general parameterizations of feasible points of LipSDP to derive various closed-form Lipschitz bounds, avoiding the use of SDP solvers. In addition, we show that our technique encompasses ECLipsE-Fast as a special case and leads to a much larger class of scalable Lipschitz bounds for deep neural networks. Our empirical study shows that our bounds improve ECLipsE-Fast, further advancing the scalability and precision of Lipschitz estimation for large neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。