用可验证代理模型，给深度学习泛化误差提供紧致且通用的上界。
Bound to Disagree: Generalization Bounds via Certifiable Surrogates
- 基于预测器间分歧构造可验证的泛化上界。
- 在无标签数据上评估，得到紧致的真风险上界。
- 不改目标模型，适配多种训练框架，适用性广。
深度学习模型的泛化界限通常空洞、不可计算或仅限特定模型类别。本文提出新的分歧型可验证证书，用于估计任意两个预测器之间真实风险的差距。通过使用具有紧密泛化保证的代理模型，并在无标签数据上评估该分歧界限，来约束目标预测器的真实风险。我们实证展示了所获证书的紧致性，并通过三种不同框架——样本压缩、模型压缩和PAC-Bayes理论——训练代理模型，验证了方法的通用性。重要的是，这些保证无需修改目标模型，也无需为泛化框架调整训练过程。
原文摘要 · Abstract (English)
Generalization bounds for deep learning models are typically vacuous, not computable or restricted to specific model classes. In this paper, we tackle these issues by providing new disagreement-based certificates for the gap between the true risk of any two predictors. We then bound the true risk of the predictor of interest via a surrogate model that enjoys tight generalization guarantees, and by evaluating our disagreement bound on an unlabeled dataset.We empirically demonstrate the tightness of the obtained certificates and showcase the versatility of the approach by training surrogate models leveraging three different frameworks: sample compression, model compression and PAC-Bayes theory. Importantly, such guarantees are achieved without modifying the target model, nor adapting the training procedure to the generalization framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。