arXiv:2411.14468cs.LGcs.AI2024-11被引 4

用分布式PID控制优化神经网络训练,提升速度与精度

A Neural Network Training Method Based on Distributed PID Control

  • 基于对称微分方程框架,用微分信号传播替代链式法则
  • 在MNIST上实现更快训练速度与更高准确率
  • 适合关注生物可解释性与控制理论融合的研究者

在前期工作中,我们提出了一种基于对称微分方程的神经网络框架,该框架具有完全对称性,具备优良的数学性质。尽管已分析部分系统数学特性,但尚未详细讨论网络训练方法。本文受传统反向传播启发,提出一种新训练范式:利用微分方程信号传播代替链式法则推导,不仅保持训练有效性,还增强生物可解释性。该方法的核心在于系统的可逆性,源于其内在对称性。然而,仅靠此法不足以实现高效训练。为此,我们进一步引入分布式比例-积分-微分(PID)控制策略,强调其在闭环系统中的实现。通过该方法,实现了更快的训练速度和更高的精度。实验在MNIST数据集上验证了其有效性,为神经网络训练提供了新视角,并拓展了控制理论的应用边界。

原文摘要 · Abstract (English)

In the previous article, we introduced a neural network framework based on symmetric differential equations. This novel framework exhibits complete symmetry, endowing it with perfect mathematical properties. While we have examined some of the system's mathematical characteristics, a detailed discussion of the network training methodology has not yet been presented. Drawing on the principles of the traditional backpropagation algorithm, this study proposes an alternative training approach that utilizes differential equation signal propagation instead of chain rule derivation. This approach not only preserves the effectiveness of training but also offers enhanced biological interpretability. The foundation of this methodology lies in the system's reversibility, which stems from its inherent symmetry,a key aspect of our research. However, this method alone is insufficient for effective neural network training. To address this, we further introduce a distributed Proportional-Integral-Derivative (PID) control approach, emphasizing its implementation within a closed system. By incorporating this method, we achieved both faster training speeds and improved accuracy. This approach not only offers novel insights into neural network training but also extends the scope of research into control methodologies. To validate its effectiveness, we apply this method to the MNIST dataset, demonstrating its practical utility.

神经网络控制理论训练优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。