模型越稳定，越需要过参数化，否则无法泛化。
The Price of Robustness: Stable Classifiers Need Overparameterization
- 用输入域到决策边界的距离衡量分类器稳定性
- 参数量接近数据量时模型必然不稳定
- 适合关注模型鲁棒性与泛化关系的研究者
连续分类器中过参数化、稳定性与泛化之间的关系尚未完全明晰。本文针对有限函数类建立了一个泛化界，该界随分类器稳定性（即输入域中到决策边界的期望距离，亦称边界裕度）提升而改善。将稳定性视为可量化的鲁棒性指标，我们推导出一个适用于不连续函数的鲁棒性定律，扩展了Bubeck与Sellke的工作，不再依赖平滑性假设。特别地，任何在n个数据点上进行插值且参数量约为n的模型必然不稳定，表明实现高稳定性需显著过参数化。对于参数化无限函数类，通过分析基于输出域裕度的更强鲁棒性度量——归一化共稳定性，我们获得类似结论。实验验证理论：模型越大越稳定，稳定性与测试性能相关，而传统范数度量则基本无预测力。
原文摘要 · Abstract (English)
The relationship between overparameterization, stability, and generalization remains incompletely understood in the setting of discontinuous classifiers. We address this gap by establishing a generalization bound for finite function classes that improves inversely with class stability, defined as the expected distance to the decision boundary in the input domain (margin). Interpreting class stability as a quantifiable notion of robustness, we derive as a corollary a law of robustness for classification that extends the results of Bubeck and Sellke beyond smoothness assumptions to discontinuous functions. In particular, any interpolating model with $p \approx n$ parameters on $n$ data points must be unstable, implying that substantial overparameterization is necessary to achieve high stability. We obtain analogous results for parameterized infinite function classes by analyzing a stronger robustness measure derived from the margin in the codomain, which we refer to as the normalized co-stability. Experiments support our theory: stability increases with model size and correlates with test performance, while traditional norm-based measures remain largely uninformative.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。