揭示鲁棒性定律与泛化误差的内在联系，解析模型复杂度与抗干扰能力的关系
Does Order Matter : Connecting The Law of Robustness to Robust Generalization
- 通过径向复杂度分析，建立鲁棒训练与测试误差的理论关联
- 证明在任意数据分布下，光滑性约束阶数为 Ω(n^{1/d})
- 发现局部区域内光滑性受扰动半径和样本浓度影响，适于研究小误差区域
Bubeck 和 Selke（2021）提出鲁棒性定律与鲁棒泛化误差之间关系的开放问题。鲁棒性定律指出，模型要实现鲁棒插值，即插值函数需满足Lipschitz连续性，必须具备过参数化特性。Wu 等人（2023）将该定律推广至任意数据分布，证明Lipschitz常数满足 $L = Ω(n^{1/d})$。鲁棒泛化问题则关注：小的鲁棒训练损失是否意味着小的鲁棒测试损失。我们利用统计学习中的Rademacher复杂度方法，建立鲁棒损失类的复杂度界，从而推导出函数类的Lipschitz性质。本文明确连接两者：（i）全局径向复杂度下，Lipschitz上界阶数保持不变；（ii）在局部尺度下，即对经验误差较小的函数子集，其光滑性上界随扰动半径 $ρ$ 与局部浓度项 $\ ext{\sqrt{r/n}}$ 变化而改变。
原文摘要 · Abstract (English)
Bubeck and Selke (2021) propose the connection between the Law of Robustness and robust generalization error as an open problem. The Law of Robustness states that overparameterization is necessary for models to interpolate robustly, i.e., the interpolating function is required to be Lipschitz. Wu et al. (2023) extend this law to arbitrary data distributions, proving that the Lipschitz constant satisfies $L = Ω(n^{1/d})$. Robust generalization, on the other hand, asks whether small robust training loss implies small robust test loss. This can be studied using statistical learning techniques such as Rademacher complexities, where a bound on the Rademacher complexity of the robust loss class implies a bound on the Lipschitzness of the function class. We use this connection to explicitly link the two for arbitrary data distributions. (i) We prove that the order of the Lipschitz bound remains the same when considering the global Rademacher complexity of robust loss classes. (ii) At the local scale, i.e., for subsets of functions with small empirical error, the order of the Lipschitz bound changes with the perturbation radius $ρ$ and the localized concentration term $\sqrt{r/n}$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。