揭示稀疏maxout网络的表达能力极限，发现深度不足时宽度无法弥补稀疏性缺陷。
On the expressivity of sparse maxout networks
- 通过虚拟多面体几何建立网络函数与结构的对偶关系
- 证明在固定输入连接数下，深度不足时宽度无法弥补表达力缺失
- 适用于研究卷积或图神经网络表达能力的理论分析
我们研究了稀疏maxout网络的表达能力，其中每个神经元从前一层固定数量的输入中选取，并使用可能具有多个参数的maxout激活函数。该设定捕捉了卷积或图神经网络的关键特性。我们建立了此类网络可计算函数与一类虚拟多面体之间的对偶关系，将网络表达能力与多面体几何联系起来。特别地，我们推导出相关多面体维度的紧致上界，成为分析的核心工具。基于此，我们构建了一系列深度层级结构。虽然足够深的稀疏maxout网络是通用的，但我们证明，若未达到所需深度，仅靠宽度无法补偿固定的输入连接数约束带来的稀疏性限制。
原文摘要 · Abstract (English)
We study the expressivity of sparse maxout networks, where each neuron takes a fixed number of inputs from the previous layer and employs a, possibly multi-argument, maxout activation. This setting captures key characteristics of convolutional or graph neural networks. We establish a duality between functions computable by such networks and a class of virtual polytopes, linking their geometry to questions of network expressivity. In particular, we derive a tight bound on the dimension of the associated polytopes, which serves as the central tool for our analysis. Building on this, we construct a sequence of depth hierarchies. While sufficiently deep sparse maxout networks are universal, we prove that if the required depth is not reached, width alone cannot compensate for the sparsity of a fixed indegree constraint.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。