破解浅层神经网络训练中的三个非凸难题，实现精确逼近与回归。
Representation and Regression Problems in Neural Networks: Relaxation, Generalization, and Numerics
- 用均值场方法将非凸问题转为凸问题，证明无松弛间隙。
- 给出泛化误差上界，指导超参数最优选择，提升测试性能。
- 提出稀疏化算法+梯度下降，高效求解高维数据的神经网络问题。
本文研究浅层神经网络在精确与近似表示、回归任务中相关的三个非凸优化问题。通过均值场方法将这些问题凸化，并利用重构定理证明不存在松弛间隙。建立了所得神经网络解的泛化误差上界，评估其在测试数据集上的预测性能，并分析关键超参数对上界的影响，从而提出最优配置方案。在计算方面，研究了凸化问题的离散化过程，推导出收敛速率。对于低维数据集，可使用单纯形法高效求解；对于高维数据集，提出一种稀疏化算法，结合过参数化浅层神经网络的梯度下降，有效求解原问题。
原文摘要 · Abstract (English)
In this work, we address three non-convex optimization problems associated with the training of shallow neural networks (NNs) for exact and approximate representation, as well as for regression tasks. Through a mean-field approach, we convexify these problems and, applying a representer theorem, prove the absence of relaxation gaps. We establish generalization bounds for the resulting NN solutions, assessing their predictive performance on test datasets and, analyzing the impact of key hyperparameters on these bounds, propose optimal choices. On the computational side, we examine the discretization of the convexified problems and derive convergence rates. For low-dimensional datasets, these discretized problems are efficiently solvable using the simplex method. For high-dimensional datasets, we propose a sparsification algorithm that, combined with gradient descent for over-parameterized shallow NNs, yields effective solutions to the primal problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。