用图结构解析ReLU网络的函数几何，揭示泛化秘密。
Discrete Functional Geometry of ReLU Networks via ReLU Transition Graphs
- 构建节点为线性区域、边为单个激活翻转的图模型
- 随机初始化时图具强扩展性，谱间隙与泛化能力相关
- 适合研究网络泛化、优化与结构设计的研究者
我们将ReLU过渡图（RTG）框架扩展为深度ReLU网络的全面图论模型。每个节点代表一个线性激活区域，边连接仅通过一个ReLU激活状态变化相连的区域，形成描述网络功能行为的离散几何结构。我们证明，在随机初始化下，RTGs表现出强扩展性、二项度分布及紧密决定泛化的谱特性。这些结构洞察使我们得以通过区域熵界定容量，并通过谱隙与逐边KL散度建立泛化上界。实验上，我们构建了小型网络的RTG，测量其平滑性与连通性，验证了理论预测：区域熵在过参数化下趋于饱和，谱隙与泛化相关，相邻区域间KL散度反映函数平滑性。本工作提供了一个统一框架，从离散函数几何视角分析ReLU网络，为理解、诊断和改进泛化提供了新工具。
原文摘要 · Abstract (English)
We extend the ReLU Transition Graph (RTG) framework into a comprehensive graph-theoretic model for understanding deep ReLU networks. In this model, each node represents a linear activation region, and edges connect regions that differ by a single ReLU activation flip, forming a discrete geometric structure over the network's functional behavior. We prove that RTGs at random initialization exhibit strong expansion, binomial degree distributions, and spectral properties that tightly govern generalization. These structural insights enable new bounds on capacity via region entropy and on generalization via spectral gap and edge-wise KL divergence. Empirically, we construct RTGs for small networks, measure their smoothness and connectivity properties, and validate theoretical predictions. Our results show that region entropy saturates under overparameterization, spectral gap correlates with generalization, and KL divergence across adjacent regions reflects functional smoothness. This work provides a unified framework for analyzing ReLU networks through the lens of discrete functional geometry, offering new tools to understand, diagnose, and improve generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。