arXiv:2608.25221cs.LGmath.OC2026-08

证明了最多8个变量的MAX函数可用两层ReLU网络精确表示

Representing MAX functions using two-hidden-layer ReLU networks

  • 通过组合双变量最大值的线性组合构造两层网络
  • 首次实现MAX₅至MAX₈的精确两层表示
  • 为理解深层网络表达能力提供新思路,适合神经网络理论研究者

本文研究使用两隐层ReLU神经网络精确表示MAXₙ(x) = max{x₁,…,xₙ}的问题。近年来,该问题被用于刻画连续分段线性函数所需的最小隐层数。目前已知下界为2,上界为logarithmic in N。目前尚不清楚最优解是否为常数层(可能仅为2层)。近期突破是[ Bakaev et al., 2026 ]中实现MAX₅的两层表示,而N≥6的情况仍开放。本文通过精心设计的计算机辅助搜索,成功获得MAX₅、MAX₆、MAX₇、MAX₈的两隐层ReLU表示。方法基于形如max{∑ᵣ max(xₐᵣ,x_bᵣ), ∑ᵣ max(x_cᵣ,x_dᵣ)}的有理线性组合,其中内层最大值在第一隐层计算,外层最大值在第二隐层完成。因此,此类形式的任意有限线性组合均可由两隐层网络实现。一旦得到MAXₙ的此类恒等式,即可获得其两隐层精确表示。最近,[Ruess et al., 2026]已对N≤10的情况给出类似表示。本文结果独立且不同,虽共享部分核心思想,但存在若干差异,或对后续研究具参考价值。

原文摘要 · Abstract (English)

We study exact representations of $\mathrm{MAX}_N(x)=\max{x_1,\ldots,x_N}$ using two-hidden-layer ReLU neural networks. This problem has been studied in recent years in an attempt to characterize the exact number of hidden layers required to represent continuous piecewise linear functions. The best lower bound is 2, while the current upper bound is logarithmic in $N$. It remains completely open if the right answer is a constant number of hidden layers (possibly even 2!) or not. In fact, a recent breakthrough was the representation of $\mathrm{MAX}_5$ as a two-hidden-layer ReLU function obtained in [Bakaev et al., 2026], and the case of $\mathrm{MAX}_N$ was stated as open for $N\geq 6$ in that paper. Using a careful computer assisted search, we obtain two-hidden-layer ReLU representations of $\mathrm{MAX}_5, \mathrm{MAX}_6, \mathrm{MAX}_7$, and $\mathrm{MAX}_8$. We obtain these by considering rational linear combinations of terms of the form $\max\{\sum_{r=1}^{s}\max(x_{a_r},x_{b_r}),\sum_{r=1}^{s}\max(x_{c_r},x_{d_r})\}$, where $a_r,b_r,c_r,d_r\in\{1,\ldots,N\}$. Each inner maximum of two coordinates can be computed in a first hidden layer, and the outer maximum of the two side-sums can be computed in a second hidden layer. Consequently, every finite linear combination of these terms has a two-hidden-layer ReLU realization. An identity for $\mathrm{MAX}_N$ in this form therefore gives an exact two-hidden-layer ReLU representation of $\mathrm{MAX}_N$. Very recently, two-hidden-layer representations of $\mathrm{MAX}_N$ of the above form were obtained for all $N\leq 10$ in [Ruess et al., 2026]. Our representations are different and were developed independently. While our techniques share most of the high-level ideas presented in [Ruess et al., 2026], there are also some minor differences which may be of interest for future research on this problem.

ReLU网络最大值函数深度表征神经网络理论

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。