用数据驱动的连续性模量评估神经网络鲁棒性,更精准反映实际表现。
Beyond Lipschitz: Data-Driven Robustness via Discrete Modulus of Continuity
- 基于离散连续性模量,不依赖模型结构,直接评估对数据分布的敏感度。
- 在ImageNet等大规模数据集上实现可扩展计算,收敛速度与数据分离距离相关。
- 能区分训练前后网络状态,揭示过拟合或欠拟合,且可得到紧致的Lipschitz估计。
神经网络的鲁棒性通常通过局部或全局Lipschitz常数衡量,但该方法可能过于粗糙或严格,无法捕捉数据相关的细微行为。本文提出一种数据驱动、架构无关的框架,基于离散模量连续性(DMOC),这是对Lipschitz连续性的非线性推广,提供更精细的鲁棒性定义。与多数现有方法不同,DMOC无需访问模型内部结构,而是相对于数据分布评估函数的规则性,将关注点从模型转向数据,以数据为基准衡量网络鲁棒性。我们建立了DMOC诱导半范数的收敛性结果,并给出了以分离距离表示的显式数据驱动收敛速率。同时引入可扩展的小批量算法,将精确计算的二次复杂度降低,使该方法适用于ImageNet等大规模数据集。实验表明,DMOC是一种独立于架构的诊断工具:能区分训练与未训练网络,揭示欠拟合和过拟合状态;作为特例,其估计的Lipschitz常数与ECLipsE及ECLipsE-fast等前沿方法相当。
原文摘要 · Abstract (English)
Robustness of neural networks is commonly quantified via local or global Lipschitz constants. However, Lipschitz continuity can be overly coarse or overly restrictive as global robustness measure, failing to capture nuanced, data-dependent behavior. We propose a data-driven, architecture-agnostic framework based on the discrete modulus of continuity (DMOC), a non linear generalization of Lipschitz continuity that provides a finer notion of robustness. Unlike many existing approaches, DMOC does not require access to model internals and instead evaluates regularity relative to the data distribution. This shifts the focus from the model to the data, which provide a data-driven baseline of regularity against which the network's robustness is assessed. We establish convergence results for DMOC-induced seminorms with explicit data-driven rates in terms of the separation distance, and introduce a scalable minibatch algorithm that reduces the quadratic cost of exact computation, enabling application to large-scale data sets such as ImageNet. Empirically, DMOC serves as an architecture independent diagnostic: it distinguishes trained from untrained networks, reveals underfitting and overfitting regimes, and yields, as a special case, tight Lipschitz estimates comparable to state-of-the-art method such as ECLipsE and ECLipsE-fast.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。