让神经网络自动学习先验，实现更灵活的贝叶斯推理与元学习。
Amortising Inference and Meta-Learning Priors in Neural Networks
- 从多个数据集学习权重先验，用变分推断实现每任务自适应
- 在极端数据稀缺下仍可完成元学习与任务内小批量训练
- 为贝叶斯神经网络提供可解释先验，适合研究生成建模与小样本学习
贝叶斯方法的核心在于根据新证据更新先验信念，但若初始无先验信念，如何维持贝叶斯框架？这是贝叶斯深度学习中的关键挑战：如何通过模型参数上的先验分布来表示对预测任务的信念。本文将贝叶斯深度学习与概率元学习相结合，提出一种从多个数据集中学习权重先验的方法，通过每任务自适应的变分推断实现。所提出的模型可视为神经过程，其潜在变量是贝叶斯神经网络（BNN）的权重，解码器则是由该潜在变量采样参数化的神经网络本身。该模型使我们能够在明确指定先验下研究贝叶斯神经网络的行为，将其作为灵活的生成模型使用，并实现此前难以达成的神经过程特性，如任务内小批量训练或极端数据稀缺下的元学习。
原文摘要 · Abstract (English)
One of the core facets of Bayesianism is in the updating of prior beliefs in light of new evidence$\text{ -- }$so how can we maintain a Bayesian approach if we have no prior beliefs in the first place? This is one of the central challenges in the field of Bayesian deep learning, where it is not clear how to represent beliefs about a prediction task by prior distributions over model parameters. Bridging the fields of Bayesian deep learning and probabilistic meta-learning, we introduce a way to $\textit{learn}$ a weights prior from a collection of datasets by introducing a way to perform per-dataset amortised variational inference. The model we develop can be viewed as a neural process whose latent variable is the set of weights of a BNN and whose decoder is the neural network parameterised by a sample of the latent variable itself. This unique model allows us to study the behaviour of Bayesian neural networks under well-specified priors, use Bayesian neural networks as flexible generative models, and perform desirable but previously elusive feats in neural processes such as within-task minibatching or meta-learning under extreme data-starvation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。