用粒子群优化训练神经网络，绕过反向传播和局部最优问题
Training neural networks without backpropagation using particles
- 分步训练单个神经元，集体协作形成网络
- 避免梯度下降陷入局部最优，提升解的质量
- 适合想探索无反向传播训练方法的研究者
神经网络由多层堆叠的神经元组成，长期依赖基于梯度下降的反向传播算法进行训练。该方法在损失函数的高维流形中易陷入多个局部极小值，且初始值会影响最终结果。粒子群优化（PSO）被提出用于在搜索空间中寻找最佳局部极小值，但其性能受限于实例化粒子的数量，难以覆盖全部解空间。本文提出新方法：将神经元单独训练，通过集体协作实现网络整体优化。该方法克服了梯度下降的局限性和PSO的搜索范围限制，提升模型求解能力。代码与数据已开源，地址为 https://github.com/dipkmr/train-nn-wobp/
原文摘要 · Abstract (English)
Neural networks are a group of neurons stacked together in multiple layers to mimic the biological neurons in a human brain. Neural networks have been trained using the backpropagation algorithm based on gradient descent strategy for several decades. Several variants have been developed to improve the backpropagation algorithm. The loss function for the neural network is optimized through backpropagation, but several local minima exist in the manifold of the constructed neural network. We obtain several solutions matching the minima. The gradient descent strategy cannot avoid the problem of local minima and gets stuck in the minima due to the initialization. Particle swarm optimization (PSO) was proposed to select the best local minima among the search space of the loss function. The search space is limited to the instantiated particles in the PSO algorithm, and sometimes it cannot select the best solution. In the proposed approach, we overcome the problem of gradient descent and the limitation of the PSO algorithm by training individual neurons separately, capable of collectively solving the problem as a group of neurons forming a network. Our code and data are available at https://github.com/dipkmr/train-nn-wobp/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。