证明了单调ReLU网络无法计算最大值函数，且深度有严格下界。
On the Depth of Monotone ReLU Neural Networks and ICNNs
- 用多面体几何与三角剖分的等周性质分析网络深度
- 证明ICNN计算n个数的最大值至少需要n层深度
- 发现深度2的ReLU网络无法被深度k的ICNN模拟
我们研究两类ReLU神经网络：单调网络（ReLU$^+$）和输入凸神经网络（ICNN）。重点探讨表达能力，特别是深度方面的限制。对于计算n个实数最大值的MAX$_n$函数，我们证明ReLU$^+$网络无法计算或近似该函数。同时，我们证明了ICNN在计算MAX$_n$时的深度复杂度存在精确的n下界。此外，还建立了ReLU网络与ICNN之间的深度分离：对任意k，存在一个大小为O(k²)的深度2的ReLU网络，无法被任何深度k的ICNN模拟。证明基于神经网络与多面体几何的深层联系，以及三角剖分的等周性质。
原文摘要 · Abstract (English)
We study two models of ReLU neural networks: monotone networks (ReLU$^+$) and input convex neural networks (ICNN). Our focus is on expressivity, mostly in terms of depth, and we prove the following lower bounds. For the maximum function MAX$_n$ computing the maximum of $n$ real numbers, we show that ReLU$^+$ networks cannot compute MAX$_n$, or even approximate it. We prove a sharp $n$ lower bound on the ICNN depth complexity of MAX$_n$. We also prove depth separations between ReLU networks and ICNNs; for every $k$, there is a depth-2 ReLU network of size $O(k^2)$ that cannot be simulated by a depth-$k$ ICNN. The proofs are based on deep connections between neural networks and polyhedral geometry, and also use isoperimetric properties of triangulations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。