多级深度学习比传统方法训练更稳定，对学习率不敏感。
Computational Advantages of Multi-Grade Deep Learning: Convergence Analysis and Performance Insights
- 采用多级结构设计模型，提升优化稳定性。
- 在图像去噪与去模糊任务中表现优于单级模型。
- 数学分析揭示其对学习率变化更具鲁棒性。
多级深度学习（MGDL）在图像回归、去噪和去模糊等任务中显著优于标准单级深度学习（SGDL）。本文研究了MGDL的计算优势，针对梯度下降（GD）方法建立了收敛性结果，并提供了性能提升的数学解释。特别地，证明了MGDL在使用GD时对学习率的选择更具鲁棒性。此外，通过分析梯度迭代中雅可比矩阵的特征值分布，揭示了MGDL训练稳定性增强的内在机制。
原文摘要 · Abstract (English)
Multi-grade deep learning (MGDL) has been shown to significantly outperform the standard single-grade deep learning (SGDL) across various applications. This work aims to investigate the computational advantages of MGDL focusing on its performance in image regression, denoising, and deblurring tasks, and comparing it to SGDL. We establish convergence results for the gradient descent (GD) method applied to these models and provide mathematical insights into MGDL's improved performance. In particular, we demonstrate that MGDL is more robust to the choice of learning rate under GD than SGDL. Furthermore, we analyze the eigenvalue distributions of the Jacobian matrices associated with the iterative schemes arising from the GD iterations, offering an explanation for MGDL's enhanced training stability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。