研究单项式激活的卷积网络几何与优化，揭示其模型表达能力与优化难点。
On the Geometry and Optimization of Polynomial Convolutional Networks
- 用代数几何方法分析网络参数映射的几何结构。
- 计算出神经流形维度与度，量化模型表达力。
- 给出大规模数据下优化临界点数量的显式公式。
我们研究具有单项式激活函数的卷积神经网络。具体而言，证明其参数化映射在滤波器缩放意义下几乎处处为正则且是同构映射。借助代数几何工具，我们探索了该映射在函数空间中的像（通常称为神经流形）的几何性质。特别地，我们计算了神经流形的维度与度，二者衡量模型的表达能力，并描述了其奇点结构。此外，对于通用的大规模数据集，我们推导出回归损失优化中临界点数量的显式公式。
原文摘要 · Abstract (English)
We study convolutional neural networks with monomial activation functions. Specifically, we prove that their parameterization map is regular and is an isomorphism almost everywhere, up to rescaling the filters. By leveraging on tools from algebraic geometry, we explore the geometric properties of the image in function space of this map - typically referred to as neuromanifold. In particular, we compute the dimension and the degree of the neuromanifold, which measure the expressivity of the model, and describe its singularities. Moreover, for a generic large dataset, we derive an explicit formula that quantifies the number of critical points arising in the optimization of a regression loss.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。