用代数方法系统分析ReLU感知机的最小化问题，发现更多隐藏解。
Algebraic Approach to Ridge-Regularized Mean Squared Error Minimization in Minimal ReLU Neural Network
- 基于代数工具构建分治枚举合并策略，精确穷举所有极小值点。
- 不仅找到孤立极小值，还发现曲线、曲面等高维极小解。
- 适合研究神经网络优化机制的理论学者，为模型理解提供新视角。
本文研究带有ReLU激活函数和岭正则化均方误差（RR-MSE）的感知机模型。由于该模型的RR-MSE在结构上为分段多项式，我们利用计算代数工具进行系统分析。具体提出一种‘分治-枚举-合并’策略，可完全枚举出所有局部极小值。借助代数形式，本方法不仅能识别数值优化通常获得的零维极小值（即孤立点），还能发现更高维的极小值（如曲线、曲面或超曲面）。尽管该代数方法对实际规模的感知机计算开销极大，但作为概念验证，我们在仅有少量隐层单元的极小感知机上成功应用了该方法。
原文摘要 · Abstract (English)
This paper investigates a perceptron, a simple neural network model, with ReLU activation and a ridge-regularized mean squared error (RR-MSE). Our approach leverages the fact that the RR-MSE for ReLU perceptron is piecewise polynomial, enabling a systematic analysis using tools from computational algebra. In particular, we develop a Divide-Enumerate-Merge strategy that exhaustively enumerates all local minima of the RR-MSE. By virtue of the algebraic formulation, our approach can identify not only the typical zero-dimensional minima (i.e., isolated points) obtained by numerical optimization, but also higher-dimensional minima (i.e., connected sets such as curves, surfaces, or hypersurfaces). Although computational algebraic methods are computationally very intensive for perceptrons of practical size, as a proof of concept, we apply the proposed approach in practice to minimal perceptrons with a few hidden units.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。