提出新泛化界，证明强局部鲁棒性可保证模型真正泛化。
Gentle Local Robustness implies Generalization
- 基于局部鲁棒性构建模型相关的新泛化界
- 在样本增多时能收敛到最优分类器的真实误差
- 适用于预训练的深层神经网络，结果非平凡
机器学习模型的鲁棒性与泛化能力在众多应用中至关重要。现有理论认为鲁棒学习算法能产生具有良好泛化能力的模型，但本文指出：现有误差界对贝叶斯最优分类器（即具有重叠类别的分类问题中最佳可测分类器）是空洞的，无法收敛至真实误差，这出人意料且此前未知。为此，我们提出一类新的、依赖于模型的误差界，其理论上优于已有鲁棒性基础界，并在样本量增加时保证收敛到最优分类器的真实误差。大量实验表明，其中两个新界在从 ImageNet 预训练的多种深度神经网络上通常是非空洞的。
原文摘要 · Abstract (English)
Robustness and generalization ability of machine learning models are of utmost importance in various application domains. There is a wide interest in efficient ways to analyze those properties. One important direction is to analyze connection between those two properties. Prior theories suggest that a robust learning algorithm can produce trained models with a high generalization ability. However, we show in this work that the existing error bounds are vacuous for the Bayes optimal classifier which is the best among all measurable classifiers for a classification problem with overlapping classes. Those bounds cannot converge to the true error of this ideal classifier. This is undesirable, surprizing, and never known before. We then present a class of novel bounds, which are model-dependent and provably tighter than the existing robustness-based ones. Unlike prior ones, our bounds are guaranteed to converge to the true error of the best classifier, as the number of samples increases. We further provide an extensive experiment and find that two of our bounds are often non-vacuous for a large class of deep neural networks, pretrained from ImageNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。