把训练好的ReLU网络看成随机仿射函数，揭示其输出分布规律。
ReLU Networks as Random Functions: Their Distribution in Probability Space
- 将ReLU网络建模为输入分布下的随机仿射函数。
- 推导出网络输出的显式概率表达式，基于高斯正卦限概率。
- 提供可计算的近似方法，定位网络能实现的仿射函数范围。
本文提出一种新框架，将训练好的ReLU网络视为由输入分布诱导的随机仿射函数。通过刻画网络激活模式的概率分布，推导出网络可实现的仿射函数的离散概率分布，并进一步描述输出的概率分布。该方法给出了基于高斯正卦限概率的显式、可数值计算的表达式。此外，我们开发了近似技术，用于确定在给定输入分布下，训练后的ReLU网络能实现的仿射函数支持集。本工作为理解含随机输入的ReLU网络的行为与性能提供了理论基础，有助于构建更可解释、更可靠的模型。
原文摘要 · Abstract (English)
This paper presents a novel framework for understanding trained ReLU networks as random, affine functions, where the randomness is induced by the distribution over the inputs. By characterizing the probability distribution of the network's activation patterns, we derive the discrete probability distribution over the affine functions realizable by the network. We extend this analysis to describe the probability distribution of the network's outputs. Our approach provides explicit, numerically tractable expressions for these distributions in terms of Gaussian orthant probabilities. Additionally, we develop approximation techniques to identify the support of affine functions a trained ReLU network can realize for a given distribution of inputs. Our work provides a framework for understanding the behavior and performance of ReLU networks corresponding to stochastic inputs, paving the way for more interpretable and reliable models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。