PAC-Bayes解释泛化能力的前提是先验能覆盖高性能预测器。
How good is PAC-Bayes at explaining generalisation?
- 基于先验分布诱导的风险分布，决定最优泛化界限。
- 若先验未覆盖高表现模型，则无法达到目标泛化水平。
- 揭示了深度学习中数据依赖先验的合理性与局限性。
我们分析了PAC-Bayes界提供有意义泛化保证的必要条件。研究表明，最优泛化保证仅取决于先验分布所诱导的风险分布。具体而言，只有当先验在高性能预测器上分配足够质量时，才能实现目标泛化水平。我们将其与深度学习中普遍采用的数据依赖先验实践相联系，并讨论了PAC-Bayes能否真正解释泛化的命题。
原文摘要 · Abstract (English)
We discuss necessary conditions for a PAC-Bayes bound to provide a meaningful generalisation guarantee. Our analysis reveals that the optimal generalisation guarantee depends solely on the distribution of the risk induced by the prior distribution. In particular, achieving a target generalisation level is only achievable if the prior places sufficient mass on high-performing predictors. We relate these requirements to the prevalent practice of using data-dependent priors in deep learning PAC-Bayes applications, and discuss the implications for the claim that PAC-Bayes ``explains'' generalisation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。