用格点规则训练神经网络，提升泛化性能并实现维度无关的误差保证。
Lattice-based Deep Neural Networks: Regularity and Tailored Regularization
- 用格点规则生成训练点，匹配目标函数的光滑性特征。
- 理论证明模型泛化误差受维数影响小，常数与输入维度无关。
- 数值实验显示新方法优于传统ℓ₂正则化，尤其在高维场景。
本文综述了格点规则在深度神经网络（DNNs）中的应用，格点规则是一类准蒙特卡洛方法，在高维积分和函数逼近中表现优异。其形式简单，仅需一个合适长度的整数生成向量即可实现。近年来，关于DNN的应用与理论研究蓬勃发展。我们回顾了近期工作：利用格点规则作为平滑激活函数的DNN训练点，获得明确的正则性界。通过约束网络参数以匹配目标函数的正则性特征，证明采用定制化格点训练点的DNN可实现良好的理论泛化误差界，且常数不依赖于输入维度。数值实验表明，使用定制正则化的DNN在性能上显著优于标准ℓ₂正则化。
原文摘要 · Abstract (English)
This survey article is concerned with the application of lattice rules to Deep Neural Networks (DNNs), lattice rules being a family of quasi-Monte Carlo methods. They have demonstrated effectiveness in various contexts for high-dimensional integration and function approximation. They are extremely easy to implement thanks to their very simple formulation -- all that is required is a good integer generating vector of length matching the dimensionality of the problem. In recent years there has been a burst of research activities on the application and theory of DNNs. We review our recent article on using lattice rules as training points for DNNs with a smooth activation function, where we obtained explicit regularity bounds of the DNNs. By imposing restrictions on the network parameters to match the regularity features of the target function, we prove that DNNs with tailored lattice training points can achieve good theoretical generalization error bounds, with implied constants independent of the input dimension. We also demonstrate numerically that DNNs trained with our tailored regularization perform significantly better than with standard $\ell_2$ regularization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。