动态调整学习率，让神经网络训练更快更稳。
Improving Neural Network Training using Dynamic Learning Rate Schedule for PINNs and Image Classification
- 根据训练损失实时调整学习率，不再固定不变。
- 在PINNs和图像分类任务中，训练速度提升且更稳定。
- 适合需要高效训练的科研与工程人员使用。
神经网络训练常因问题复杂而困难，即使使用更深更宽的网络，错误的超参数选择仍会带来挑战。学习率是关键超参数，传统方法通常保持静态。针对复杂系统中梯度变化多变的问题，本文提出一种基于训练过程损失值动态调整学习率的算法（DLRS）。实验在物理信息神经网络（PINNs）和图像分类任务上展开，分别采用多层感知机和卷积神经网络。结果表明，所提方法能显著加速训练并提升稳定性。
原文摘要 · Abstract (English)
Training neural networks can be challenging, especially as the complexity of the problem increases. Despite using wider or deeper networks, training them can be a tedious process, especially if a wrong choice of the hyperparameter is made. The learning rate is one of such crucial hyperparameters, which is usually kept static during the training process. Learning dynamics in complex systems often requires a more adaptive approach to the learning rate. This adaptability becomes crucial to effectively navigate varying gradients and optimize the learning process during the training process. In this paper, a dynamic learning rate scheduler (DLRS) algorithm is presented that adapts the learning rate based on the loss values calculated during the training process. Experiments are conducted on problems related to physics-informed neural networks (PINNs) and image classification using multilayer perceptrons and convolutional neural networks, respectively. The results demonstrate that the proposed DLRS accelerates training and improves stability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。