用泊松超平面过程建模两层ReLU网络，实现可扩展的贝叶斯推理
Poisson Hyperplane Processes with Rectified Linear Units
- 将高斯先验下的泊松超平面过程作为两层ReLU网络的概率替代表示
- 通过分解性质实现模型在大规模问题上的可扩展性
- 提出退火序蒙特卡洛算法，实验显示性能优于经典两层ReLU网络
神经网络在各类分类与回归任务中表现优异。修正线性单元（ReLU）常被用作神经网络隐藏层的激活函数。本文建立了泊松超平面过程（PHP）与两层ReLU神经网络之间的联系，证明了具有高斯先验的PHP是两层ReLU神经网络的一种替代概率表示。此外，我们展示了基于PHP构建的两层神经网络可通过分解命题实现大规模问题的可扩展性。最后，提出了用于贝叶斯推断的退火序蒙特卡洛算法。数值实验表明，所提方法性能优于经典两层ReLU神经网络。模型实现代码已公开于 https://github.com/ShufeiGe/Pois_Relu.git。
原文摘要 · Abstract (English)
Neural networks have shown state-of-the-art performances in various classification and regression tasks. Rectified linear units (ReLU) are often used as activation functions for the hidden layers in a neural network model. In this article, we establish the connection between the Poisson hyperplane processes (PHP) and two-layer ReLU neural networks. We show that the PHP with a Gaussian prior is an alternative probabilistic representation to a two-layer ReLU neural network. In addition, we show that a two-layer neural network constructed by PHP is scalable to large-scale problems via the decomposition propositions. Finally, we propose an annealed sequential Monte Carlo algorithm for Bayesian inference. Our numerical experiments demonstrate that our proposed method outperforms the classic two-layer ReLU neural network. The implementation of our proposed model is available at https://github.com/ShufeiGe/Pois_Relu.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。