揭示神经网络理性激活函数的复杂度与学习理论的深层联系
Erzeugunsgrad, VC-Dimension and Neural Networks with rational activation function
- 用构造集次数扩展生成度概念，建立代数几何与学习理论桥梁
- 证明VC维与克鲁尔维数在对数因子下线性相关
- 适用于分析带理性激活函数的神经网络泛化能力
Joos Heintz于1983年提出生成度（Erzeugungsgrad）以界定量化消除后非空胞腔的数量。本文基于Pardo-Sebastián（2022）定义的构造集次数，扩展该概念及Heintz（1983）定理中的组合界。我们证明生成度是连接代数闭域上的仿射交比理论与参数化构造集族的分类器所对应的计算学习理论中VC理论的关键要素。特别地，建立了VC维与克鲁尔维数之间的线性关系（含对数因子）。利用此关系，研究了在回避簇中正确测试序列的密度。最后将这些思想应用于具有理性激活函数的参数化神经网络族的分析。
原文摘要 · Abstract (English)
The notion of Erzeugungsgrad was introduced by Joos Heintz in 1983 to bound the number of non-empty cells occurring after a process of quantifier elimination. We extend this notion and the combinatorial bounds of Theorem 2 in Heintz (1983) using the degree for constructible sets defined in Pardo-Sebastián (2022). We show that the Erzeugungsgrad is the key ingredient to connect affine Intersection Theory over algebraically closed fields and the VC-Theory of Computational Learning Theory for families of classifiers given by parameterized families of constructible sets. In particular, we prove that the VC-dimension and the Krull dimension are linearly related up to logarithmic factors based on Intersection Theory. Using this relation, we study the density of correct test sequences in evasive varieties. We apply these ideas to analyze parameterized families of neural networks with rational activation function.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。