用代数几何方法解析ReLU网络能表示哪些函数。
Toric geometry of ReLU neural networks
- 将ReLU网络与环面几何关联，构建对应的扇形和除子结构。
- 给出浅层网络可实现函数的充要条件，基于交数计算。
- 为理解神经网络表达能力提供全新几何视角，适合理论研究者。
给定一个连续的、有限段线性函数 $f:\mathbb{R}^{n_0} \to \mathbb{R}$ 及固定架构 $(n_0,\ldots,n_k;1)$ 的前馈ReLU神经网络，精确函数实现问题旨在判断是否存在该架构的网络能够实现 $f$。为系统回答此类问题，本文建立环面几何与ReLU神经网络之间的联系，使代数几何中的诸多结构与工具可用于研究神经网络。针对具有有理权重的无偏置ReLU网络，定义了ReLU扇形、ReLU环面簇及与网络相关的ReLUCartier除子。该工作还揭示了热带几何与环面几何在ReLU网络中的关联。作为应用，通过计算ReLU Cartier除子与环不变曲线的交数，证明了无偏置单层ReLU网络可实现函数的充要条件。
原文摘要 · Abstract (English)
Given a continuous finitely piecewise linear function $f:\mathbb{R}^{n_0} \to \mathbb{R}$ and a fixed architecture $(n_0,\ldots,n_k;1)$ of feedforward ReLU neural networks, the exact function realization problem is to determine when some network with the given architecture realizes $f$. To develop a systematic way to answer these questions, we establish a connection between toric geometry and ReLU neural networks. This approach enables us to utilize numerous structures and tools from algebraic geometry to study ReLU neural networks. Starting with an unbiased ReLU neural network with rational weights, we define the ReLU fan, the ReLU toric variety, and the ReLU Cartier divisor associated with the network. This work also reveals the connection between the tropical geometry and the toric geometry of ReLU neural networks. As an application of the toric geometry framework, we prove a necessary and sufficient criterion of functions realizable by unbiased shallow ReLU neural networks by computing intersection numbers of the ReLU Cartier divisor and torus-invariant curves.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。