arXiv:2504.09107cs.LGcs.NE2025-04

提出一种收缩初始化方法,提升神经网络训练的稳定性与平滑性。

Shrinkage Initialization for Smooth Learning of Neural Networks

  • 采用收缩策略初始化各层权重,适配任意结构的神经网络。
  • 在多个数据集上实现稳定且鲁棒的训练效果。
  • 适合追求训练平稳性和通用初始化方案的研究者。

智能系统的成功高度依赖于信息的人工学习,推动了神经学习方案的广泛应用。尽管已有基于层的初始化方法,但通用的初始阶段解决方案仍待探索。本文提出一种改进的神经学习初始化方法,通过收缩策略初始化网络每层的变换参数,可普遍适用于具有随机层数的网络结构,并实现稳定性能。同时引入平滑学习机制,以应对神经学习中的多样性影响。在多个人工数据集上的实验表明,该方法在收缩初始化下表现稳健,具备良好的神经网络平滑学习能力。

原文摘要 · Abstract (English)

The successes of intelligent systems have quite relied on the artificial learning of information, which lead to the broad applications of neural learning solutions. As a common sense, the training of neural networks can be largely improved by specifically defined initialization, neuron layers as well as the activation functions. Though there are sequential layer based initialization available, the generalized solution to initial stages is still desired. In this work, an improved approach to initialization of neural learning is presented, which adopts the shrinkage approach to initialize the transformation of each layer of networks. It can be universally adapted for the structures of any networks with random layers, while stable performance can be attained. Furthermore, the smooth learning of networks is adopted in this work, due to the diverse influence on neural learning. Experimental results on several artificial data sets demonstrate that, the proposed method is able to present robust results with the shrinkage initialization, and competent for smooth learning of neural networks.

神经网络初始化平滑学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。