给出多层感知机的完整批矩阵形式推导与实现,让算法更透明可优化。
Batch Matrix-form Equations and Implementation of Multilayer Perceptrons
- 用批矩阵形式完整推导MLP前向与反向传播
- 通过SymPy符号验证所有梯度公式正确性
- 提供跨平台高效实现,特别支持稀疏计算
多层感知机(MLPs)是现代深度学习的基础,但其算法细节通常未以完整的批矩阵形式呈现,多数文献仅按样本表达梯度或依赖自动微分。尽管自动微分能达高效率,但批矩阵形式使计算结构清晰可见,对稀疏神经网络等场景下的透明分析与优化至关重要。本文填补这一空白,提供数学严谨且可直接实现的MLP批矩阵形式规范。推导了所有标准及高级层(包括批归一化、Softmax)的前向与反向方程,并使用SymPy符号数学库验证所有梯度公式。基于这些规范,构建了在NumPy、PyTorch、JAX、TensorFlow和高性能C++后端中的统一参考实现。主要贡献包括:(1) 完整推导MLP的批矩阵形式反向传播;(2) 所有梯度方程的符号验证;(3) 基于少量矩阵原语的统一Python与C++实现;(4) 展示显式形式如何支持高效稀疏计算。这些成果建立了一个可验证、可扩展的神经网络算法理解、教学与研究基础。
原文摘要 · Abstract (English)
Multilayer perceptrons (MLPs) remain fundamental to modern deep learning, yet their algorithmic details are rarely presented in complete, explicit \emph{batch matrix-form}. Rather, most references express gradients per sample or rely on automatic differentiation. Although automatic differentiation can achieve equally high computational efficiency, the usage of batch matrix-form makes the computational structure explicit, which is essential for transparent, systematic analysis, and optimization in settings such as sparse neural networks. This paper fills that gap by providing a mathematically rigorous and implementation-ready specification of MLPs in batch matrix-form. We derive forward and backward equations for all standard and advanced layers, including batch normalization and softmax, and validate all equations using the symbolic mathematics library SymPy. From these specifications, we construct uniform reference implementations in NumPy, PyTorch, JAX, TensorFlow, and a high-performance C++ backend optimized for sparse operations. Our main contributions are: (1) a complete derivation of batch matrix-form backpropagation for MLPs, (2) symbolic validation of all gradient equations, (3) uniform Python and C++ reference implementations grounded in a small set of matrix primitives, and (4) demonstration of how explicit formulations enable efficient sparse computation. Together, these results establish a validated, extensible foundation for understanding, teaching, and researching neural network algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。