研究神经网络在输入扰动下的不确定性传播，给出输出概率密度的解析解。
Uncertainty propagation in feed-forward neural network models
- 通过线性化漏失ReLU激活函数，推导输出分布的解析表达式。
- 理论预测与蒙特卡洛模拟结果高度一致，误差极小。
- 适合关注模型可靠性与不确定性量化的研究者参考。
我们为带有漏失ReLU激活函数的前馈神经网络，在输入向量存在随机扰动的情况下，开发了新的不确定性传播方法。具体而言，推导出神经网络输出的概率密度函数（PDF）及其统计矩关于输入不确定性及网络参数（权重和偏置）的解析表达式。关键发现是：对漏失ReLU进行适当线性化，即使在较大的输入扰动下也能获得精确的统计结果，这归因于信息在网络中的传播机制。此外，我们提出新型可解析处理的高斯拷贝替代模型，用于逼近神经网络输出的联合概率密度函数。为验证理论结果，我们在一个表示多项式函数空间间非线性积分微分算子的多层神经网络上进行了蒙特卡洛仿真与详细的误差分析。结果表明，理论预测与仿真之间表现出极佳的一致性。
原文摘要 · Abstract (English)
We develop new uncertainty propagation methods for feed-forward neural network architectures with leaky ReLU activation functions subject to random perturbations in the input vectors. In particular, we derive analytical expressions for the probability density function (PDF) of the neural network output and its statistical moments as a function of the input uncertainty and the parameters of the network, i.e., weights and biases. A key finding is that an appropriate linearization of the leaky ReLU activation function yields accurate statistical results even for large perturbations in the input vectors. This can be attributed to the way information propagates through the network. We also propose new analytically tractable Gaussian copula surrogate models to approximate the full joint PDF of the neural network output. To validate our theoretical results, we conduct Monte Carlo simulations and a thorough error analysis on a multi-layer neural network representing a nonlinear integro-differential operator between two polynomial function spaces. Our findings demonstrate excellent agreement between the theoretical predictions and Monte Carlo simulations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。