将随机预测的泛化保证转化为确定性模型的可靠边界。
A Framework for Bounding Deterministic Risk with PAC-Bayes: Applications to Majority Votes
- 从随机假设的泛化界推导出单个确定性模型的边界
- 在多数投票分类器上实测性能优于基线最高2倍
- 适用于需部署单一确定性模型的实用场景
PAC-Bayes 是一种广泛使用且高效的框架，可在不可数假设空间中提供泛化保证。然而，其经典形式仅能给出随机采样假设的期望风险保证，这要求测试时进行随机预测，导致在必须部署单一确定性假设的许多实际场景中无法应用。本文提出一个统一框架，可从随机 PAC-Bayesian 保证中提取针对单个假设的保证。我们给出一个通用的最优界，并由此导出一个数值界及对多数投票的特化结果。实验表明，该方法在确定性分类器的泛化界估计上始终优于主流基线，性能提升可达两倍。
原文摘要 · Abstract (English)
PAC-Bayes is a popular and efficient framework for obtaining generalization guarantees in situations involving uncountable hypothesis spaces. Unfortunately, in its classical formulation, it only provides guarantees on the expected risk of a randomly sampled hypothesis. This requires stochastic predictions at test time, making PAC-Bayes unusable in many practical situations where a single deterministic hypothesis must be deployed. We propose a unified framework to extract guarantees holding for a single hypothesis from stochastic PAC-Bayesian guarantees. We present a general oracle bound and derive from it a numerical bound and a specialization to majority vote. We empirically show that our approach consistently outperforms popular baselines (by up to a factor of 2) when it comes to generalization bounds on deterministic classifiers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。