arXiv:2607.11289cs.NEcs.LG2026-07

将反向传播重新定义为一个可精确求解的线性系统,揭示其数学本质。

Backpropagation as a Nilpotent Linear System

  • 用全局算子理论将反向传播重构为单一线性系统 (I−B)X=G
  • 证明反向算子B是幂零矩阵,解在最多L步内终止
  • 适用于理解残差网络与迁移学习的梯度机制

反向传播是深度学习的核心计算引擎,但其数学结构常被视为计算图的逐层遍历过程。本文提出全局算子理论下的F-伴随框架,将深度为L的前馈网络的逐层反向递归转化为单一线性系统 (I−B)X=G,其中G为源向量。我们证明全局反向算子B严格块上三角且幂零指数至多为L。该幂零性保证了诺伊曼级数解在最多L项后精确终止,揭示经典反向传播在数学上等价于上双对角系统的块回代。我们形式化了F-对称性——反向过程与正向过程完全镜像的条件,识别出正交权矩阵为典型例子。通过数值实例,我们展示该算子视角如何揭示严格前馈网络的单路径坍缩及其在残差架构中的破坏。最后,利用这一组合结构,我们严谨推导了残差网络(梯度高速公路)与迁移学习(梯度截断)的机制。该框架将反向传播从算法配方提升为全局幂零算子表述。

原文摘要 · Abstract (English)

Backpropagation is the computational engine of deep learning, yet its mathematical structure is typically treated as a procedural traversal of computational graphs. We present a global operator theory of the \emph{F-adjoint} framework, which reformulates the layerwise backward recursion of an $L$-depth feedforward network into a single linear system $(I-\cB)\Xs=\bG$, where $\bG$ is a source vector. We prove that the global backward operator $\cB$ is strictly block upper-triangular and nilpotent of index at most $L$. This nilpotency guarantees the exact termination of the Neumann series solution after at most $L$ terms, revealing classical backpropagation to be mathematically equivalent to block back-substitution on an upper bidiagonal system. We formalise \emph{F-symmetry} -- the condition in which the backward pass perfectly mirrors the forward pass -- identifying orthogonal weight matrices as canonical examples. Through worked numerical examples, we demonstrate how this operator perspective exposes the single-path collapse of strictly feedforward networks and its breakdown in residual architectures. Finally, we leverage this compositional structure to rigorously derive the mechanics of residual networks (gradient highways) and transfer learning (gradient truncation). This framework elevates backpropagation from an algorithmic recipe to a global nilpotent-operator formulation.

反向传播幂零性线性系统深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。