破解单隐层ReLU网络的黑箱机制，让训练结果可解释。
On the Principles of ReLU Networks with One Hidden Layer
- 构建通用函数逼近解，揭示单隐层网络的数学机理。
- 一维输入下训练解可完全解析，高维也能部分解释。
- 为深层ReLU网络理解提供理论基础，适合研究者参考。
具有一个隐藏层的两层神经网络是最简单的前馈网络，其机制可能构成更复杂架构的基础。然而，这类简单结构仍被视为‘黑箱’——难以解释反向传播得到的解的内在机制，也无法通过确定性方法控制训练过程。本文系统研究了该问题，通过构造通用函数逼近解，证明在一维输入情况下，训练解可被完全理解；在高维输入下也可在一定程度上被有效解释。这些成果为彻底揭开两层ReLU网络的黑箱之谜奠定了基础，推动了对深层ReLU网络的理解。
原文摘要 · Abstract (English)
A neural network with one hidden layer or a two-layer network (regardless of the input layer) is the simplest feedforward neural network, whose mechanism may be the basis of more general network architectures. However, even to this type of simple architecture, it is also a ``black box''; that is, it remains unclear how to interpret the mechanism of its solutions obtained by the back-propagation algorithm and how to control the training process through a deterministic way. This paper systematically studies the first problem by constructing universal function-approximation solutions. It is shown that, both theoretically and experimentally, the training solution for the one-dimensional input could be completely understood, and that for a higher-dimensional input can also be well interpreted to some extent. Those results pave the way for thoroughly revealing the black box of two-layer ReLU networks and advance the understanding of deep ReLU networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。