揭示了ReLU网络线性区域的几何结构，发现其连接图性质与网络深度宽度无关。
Characterizing the Discrete Geometry of ReLU Networks
- 构建线性区域间的连接图，分析其拓扑特性
- 平均度上限为输入维度的两倍，与网络规模无关
- 图直径有界，适合研究网络泛化与稳定性
ReLU网络定义连续的分段线性函数，其线性区域在输入空间中构成多面体复形，完全划分输入空间。这些区域的连接方式决定了网络行为，非线性仅出现在区域边界。然而，对这类复形几何的理解仍有限，精确计算复杂度极高。本文针对全连接ReLU网络，证明了其区域连接图的新理论性质：节点代表区域，边表示共享面的区域相连。结果表明，该图的平均度不超过输入维度的两倍，与网络宽度和深度无关；图直径存在上界，不随输入维度增长，尽管区域数量随输入维度指数级增加。实验验证了上述结论，使用合成数据和真实数据训练的网络均表现出一致规律。代码开源于 https://github.com/bl-ake/ICLR-2026。
原文摘要 · Abstract (English)
It is well established that ReLU networks define continuous piecewise-linear functions, and that their linear regions are polyhedra in the input space. These regions form a complex that fully partitions the input space. The way these regions fit together is fundamental to the behavior of the network, as nonlinearities occur only at the boundaries where these regions connect. However, relatively little is known about the geometry of these complexes beyond bounds on the total number of regions, and calculating the complex exactly is intractable for most networks. In this work, we prove new theoretical results about these complexes that hold for all fully-connected ReLU networks, specifically about their connectivity graphs in which nodes correspond to regions and edges exist between each pair of regions connected by a face. We find that the average degree of this graph is upper bounded by twice the input dimension regardless of the width and depth of the network, and that the diameter of this graph has an upper bound that does not depend on input dimension, despite the number of regions increasing exponentially with input dimension. We corroborate our findings through experiments with networks trained on both synthetic and real-world data, which provide additional insight into the geometry of ReLU networks. Code to reproduce our results can be found at https://github.com/bl-ake/ICLR-2026.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。