揭示了ReLU网络输出的代数结构，为理解其表达能力提供新视角。
Constraining the outputs of ReLU neural networks
- 基于ReLU网络的分段线性特性，构建其输出对应的代数簇。
- 推导出刻画网络可表示函数的多项式方程约束。
- 揭示网络结构在特定条件下达到预期维度的机制，适合理论研究者。
我们引入一类与ReLU神经网络自然相关的代数簇，源于其输出在输入空间激活区域内的分段线性结构，以及在参数空间中的分段多重线性结构。通过分析每个激活区域内网络输出的秩约束，我们推导出刻画网络可表示函数的多项式方程。进一步研究了这些代数簇达到其期望维度的条件，为理解ReLU网络的表达能力和结构特性提供了深入见解。
原文摘要 · Abstract (English)
We introduce a class of algebraic varieties naturally associated with ReLU neural networks, arising from the piecewise linear structure of their outputs across activation regions in input space, and the piecewise multilinear structure in parameter space. By analyzing the rank constraints on the network outputs within each activation region, we derive polynomial equations that characterize the functions representable by the network. We further investigate conditions under which these varieties attain their expected dimension, providing insight into the expressive and structural properties of ReLU networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。