用奇异学习理论推导出可计算的贝叶斯泛化界，适用于过参数模型。
PAC-Bayes Bounds for Gibbs Posteriors via Singular Learning Theory
- 基于奇异学习理论分析吉布斯后验的积分项，获得数据自适应的泛化界。
- 在低秩矩阵补全和ReLU网络上，新界比经典复杂度界更紧。
- 适合研究过参数模型泛化性的研究人员，尤其关注理论边界者。
我们推导了吉布斯后验的显式非渐近PAC-Bayes泛化界，即通过经验风险对先验进行指数倾斜得到的数据相关参数分布。不同于依赖度量熵控制的古典最坏情况复杂度界，我们的分析得到的是后验平均风险界，适用于过参数化模型，并能适应数据结构与内在模型复杂度。该界包含参数空间上的边际型积分，我们借助奇异学习理论对其进行分析，获得后验风险的显式且具有实际意义的刻画。在低秩矩阵补全、ReLU神经网络回归与分类中的应用表明，所得界在理论上可计算且显著优于经典复杂度基界。结果凸显了PAC-Bayes分析在现代过参数化与奇异模型中实现精确有限样本泛化保证的潜力。
原文摘要 · Abstract (English)
We derive explicit non-asymptotic PAC-Bayes generalization bounds for Gibbs posteriors, that is, data-dependent distributions over model parameters obtained by exponentially tilting a prior with the empirical risk. Unlike classical worst-case complexity bounds based on uniform laws of large numbers, which require explicit control of the model space in terms of metric entropy (integrals), our analysis yields posterior-averaged risk bounds that can be applied to overparameterized models and adapt to the data structure and the intrinsic model complexity. The bound involves a marginal-type integral over the parameter space, which we analyze using tools from singular learning theory to obtain explicit and practically meaningful characterizations of the posterior risk. Applications to low-rank matrix completion and ReLU neural network regression and classification show that the resulting bounds are analytically tractable and substantially tighter than classical complexity-based bounds. Our results highlight the potential of PAC-Bayes analysis for precise finite-sample generalization guarantees in modern overparameterized and singular models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。