对比五种优化算法，指导深度学习训练中选型与调参。
Gradient Descent Algorithm Survey
- 系统分析SGD、Mini-batch SGD、Momentum、Adam、Lion的优劣。
- 给出各算法在实际场景中的配置建议和适用条件。
- 适合研究者与工程师快速选择合适优化器并调优参数。
针对深度学习中优化算法的实际配置需求，本文聚焦于SGD、小批量SGD、Momentum、Adam和Lion五种主流算法，系统分析其核心优势、局限性及关键实用建议。研究旨在深入理解这些算法，并为学术研究与工程实践中的合理选型、参数调优与性能提升提供标准化参考，助力解决不同规模模型及多样训练场景下的优化挑战。
原文摘要 · Abstract (English)
Focusing on the practical configuration needs of optimization algorithms in deep learning, this article concentrates on five major algorithms: SGD, Mini-batch SGD, Momentum, Adam, and Lion. It systematically analyzes the core advantages, limitations, and key practical recommendations of each algorithm. The research aims to gain an in-depth understanding of these algorithms and provide a standardized reference for the reasonable selection, parameter tuning, and performance improvement of optimization algorithms in both academic research and engineering practice, helping to solve optimization challenges in different scales of models and various training scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。