用最小二乘法训练两层二次激活的卷积网络，实现全局最优且可解析推导输入输出关系。
Least Squares Training of Quadratic Convolutional Neural Networks with Applications to System Theory
- 基于最小二乘法求解两层二次激活网络的全局最优权重
- 训练速度显著提升，预测精度损失小，且得到闭式输入输出表达式
- 适合需要可分析性与安全性的系统理论场景，如自动驾驶
本文为使用二次激活函数、2-范数损失且无正则化的两层卷积神经网络提供了最小二乘训练方法。该方法获得了全局最优权重的解析表达式，并导出了网络的二次输入-输出方程。这一特性使网络在系统理论中具有应用价值，支持对输入扰动敏感性的进一步分析，对航空器或自动驾驶等安全关键系统至关重要。方法与以往二次网络训练策略及反向传播训练的ReLU网络对比，结果表明所提方法训练时间大幅缩短，预测精度损失微小，且具备解析输入输出关系的优势。该方法应用于系统辨识和GPS位置估计任务，虽仅适用于两层网络，但为深入研究更深的二次网络在系统理论中的应用提供了启发。
原文摘要 · Abstract (English)
This paper provides a least squares formulation for the training of a 2-layer convolutional neural network using quadratic activation functions, a 2-norm loss function, and no regularization term. Using this method, an analytic expression for the globally optimal weights is obtained alongside a quadratic input-output equation for the network. These properties make the network a viable tool in system theory by enabling further analysis, such as the sensitivity of the output to perturbations in the input, which is crucial for safety-critical systems such as aircraft or autonomous vehicles. The least squares method is compared to previously proposed strategies for training quadratic networks and to a back-propagation-trained ReLU network. The proposed method is applied to a system identification problem and a GPS position estimation problem. The least squares network is shown to have a significantly reduced training time with minimal compromises on prediction accuracy alongside the advantages of having an analytic input-output equation. Although these results only apply to 2-layer networks, this paper motivates the exploration of deeper quadratic networks in the context of system theory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。