用元学习超网络构建神经网络泛化边界，提升模型可靠性。
Generalization Bounds via Meta-Learned Model Representations: PAC-Bayes and Sample Compression Hypernetworks
- 设计三种编码器：基于PAC-Bayes的后验分布、样本压缩选样+离散消息、混合连续消息编码。
- 提出新样本压缩定理，利用编码-解码间信息传递计算下游模型泛化界。
- 适用于需要可证明泛化性能的高可靠性场景，如医疗或金融建模。
PAC-Bayesian和样本压缩学习框架对于推导神经网络的紧致（非空洞）泛化边界至关重要。本文在元学习框架中利用这些结果，采用一个超网络，从输入数据集生成下游预测器的参数。方法的创新在于对超网络架构的设计：(1) 基于PAC-Bayes的编码器，用于表达潜在空间上的后验分布；(2) 样本压缩编码器，从输入数据集中选择一小部分样本并携带来自离散集合的消息；(3) 结合两者的新混合架构，其理论基础是新的样本压缩定理，能处理连续消息。该定理利用编码器与解码器之间的关键信息传递，为通过元学习获得的每个下游预测器提供泛化保证。
原文摘要 · Abstract (English)
Both PAC-Bayesian and Sample Compress learning frameworks are instrumental for deriving tight (non-vacuous) generalization bounds for neural networks. We leverage these results in a meta-learning scheme, relying on a hypernetwork that outputs the parameters of a downstream predictor from a dataset input. The originality of our approach lies in the investigated hypernetwork architectures that encode the dataset before decoding the parameters: (1) a PAC-Bayesian encoder that expresses a posterior distribution over a latent space, (2) a Sample Compress encoder that selects a small sample of the dataset input along with a message from a discrete set, and (3) a hybrid between both approaches motivated by a new Sample Compress theorem handling continuous messages. The latter theorem exploits the pivotal information transiting at the encoder-decoder junction in order to compute generalization guarantees for each downstream predictor obtained by our meta-learning scheme.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。