arXiv:2510.25042cs.LGcs.NE2025-10

动态调整动量与学习率,提升深度网络训练效率

Dynamically Weighted Momentum with Adaptive Step Sizes for Efficient Deep Network Training

  • 基于历史数据动态调节动量和学习率
  • 在多种场景下实现更快收敛和更高精度
  • 适合复杂模型与非凸优化任务

当前深度学习研究中,尽管SGD和Adam等优化算法已广泛应用,但仍难以应对学习效率波动、复杂模型需求及非凸优化问题。主要瓶颈在于对复杂数据结构与高维空间的适应能力不足,如学习率选择困难、易陷入局部最优等。为此,本文提出新型优化算法DWMGrad,基于传统方法构建,引入依赖历史数据的动态引导机制,实现动量与学习率的自适应更新。该策略使优化器能灵活调整对历史信息的依赖,适应不同训练场景。大量实验验证表明,DWMGrad在多种条件下均具备更快收敛速度和更高准确率。

原文摘要 · Abstract (English)

Within the current sphere of deep learning research, despite the extensive application of optimization algorithms such as Stochastic Gradient Descent (SGD) and Adaptive Moment Estimation (Adam), there remains a pronounced inadequacy in their capability to address fluctuations in learning efficiency, meet the demands of complex models, and tackle non-convex optimization issues. These challenges primarily arise from the algorithms' limitations in handling complex data structures and models, for instance, difficulties in selecting an appropriate learning rate, avoiding local optima, and navigating through high-dimensional spaces. To address these issues, this paper introduces a novel optimization algorithm named DWMGrad. This algorithm, building on the foundations of traditional methods, incorporates a dynamic guidance mechanism reliant on historical data to dynamically update momentum and learning rates. This allows the optimizer to flexibly adjust its reliance on historical information, adapting to various training scenarios. This strategy not only enables the optimizer to better adapt to changing environments and task complexities but also, as validated through extensive experimentation, demonstrates DWMGrad's ability to achieve faster convergence rates and higher accuracies under a multitude of scenarios.

优化算法深度学习自适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。