有限VC维网络在逼近与学习间存在权衡，深度影响其性能表现。
Networks with Finite VC Dimension: Pro and Contra
- 基于高维几何的集中性，有限VC维使误差趋于确定性
- 小VC维利于学习一致性，但可能削弱对分布函数的逼近能力
- 深度ReLU网络在精度与一致性间存在关键权衡
本文从高维几何与统计学习理论视角，研究神经网络在大规模数据集上分类器的逼近与学习问题。比较了网络输入输出函数集合的VC维对逼近能力与样本学习一致性的双重影响。研究表明：尽管有限VC维有利于经验误差的统一收敛，却未必利于从建模实际应用中出现概率分布的函数中进行逼近。基于高维几何的集中性原理，证明当网络实现具有有限VC维的输入输出函数集时，其逼近误差与经验误差在处理大规模数据时几乎呈现确定性行为。讨论了通用逼近性质的实际局限，以及逼近精度与学习一致性之间的权衡，并分析了含ReLU单元的深层网络深度对其精度与一致性的影响。
原文摘要 · Abstract (English)
Approximation and learning of classifiers of large data sets by neural networks in terms of high-dimensional geometry and statistical learning theory are investigated. The influence of the VC dimension of sets of input-output functions of networks on approximation capabilities is compared with its influence on consistency in learning from samples of data. It is shown that, whereas finite VC dimension is desirable for uniform convergence of empirical errors, it may not be desirable for approximation of functions drawn from a probability distribution modeling the likelihood that they occur in a given type of application. Based on the concentration-of-measure properties of high dimensional geometry, it is proven that both errors in approximation and empirical errors behave almost deterministically for networks implementing sets of input-output functions with finite VC dimensions in processing large data sets. Practical limitations of the universal approximation property, the trade-offs between the accuracy of approximation and consistency in learning from data, and the influence of depth of networks with ReLU units on their accuracy and consistency are discussed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。