提出带依赖权重的贝叶斯神经网络，突破传统高斯先验局限。
Posterior Bayesian Neural Networks with Dependent Weights
- 用依赖权重和重尾分布建模网络，改进传统高斯先验
- 在宽层极限下，后验输出收敛为可逆协方差的高斯混合
- 给出激活函数与莱维测度的充分条件，确保顺序无关性
本文研究具有依赖且可能重尾权重的全连接前馈深度神经网络，以克服标准高斯先验的局限。已有研究表明，当隐藏层节点数趋于无穷时，按顺序极限方式，输出分布弱收敛于高斯混合。本文从高斯似然下的后验分布视角出发，若先验下无限宽极限的随机协方差矩阵正定，则在顺序极限下可确定输出的后验分布。我们给出了保证该协方差矩阵可逆的温和充分条件，从而扩展了[8]的结果。此外，我们提出了关于模型参数（激活函数及其关联的莱维测度）的充分条件，使得顺序极限与排列顺序无关。通过具体例子和数值模拟验证了结论。
原文摘要 · Abstract (English)
We consider fully connected and feedforward deep neural networks with dependent and possibly heavy-tailed weights, as introduced in [26], to address limitations of the standard Gaussian prior. It has been proved in [26] that, as the number of nodes in the hidden layers grows large, according to a sequential and ordered limit, the law of the output converges weakly to a Gaussian mixture. In this paper, we study the neural network through the lens of the posterior distribution with a Gaussian likelihood. If the random covariance matrix of the infinite-width limit is positive definite under the prior, we identify the posterior distribution of the output in the wide-width limit according to a sequential regime. Remarkably, we provide mild sufficient conditions to ensure the aforementioned invertibility of the random covariance matrix under the prior, thereby extending the results in [8]. Among our results, we present sufficient conditions on some model parameters (the activation function and the associated Lévy measures) which ensure that the sequential limits are independent of the order. We illustrate our findings with examples and numerical simulations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。