系统评估学习率调度对多种模型的影响，发现调度策略效果因架构而异。
Systematic Evaluation of Learning Rate Scheduling Strategies Across Heterogeneous Architectures
- 在30种不同架构上测试25种调度策略，自动注入代码批量训练
- 最高准确率达86.45%，237个模型超80%准确率
- 余弦退火和循环调度优于基础衰减，适合高效选型参考
选择学习率调度策略对神经网络训练至关重要，但人工筛选成本高且不全面。我们系统研究了其在多样化架构上的影响，评估了来自卷积和Transformer家族的30种代表性架构，基于LEMUR神经网络数据集。通过自动化源码注入，在九个PyTorch调度族中应用25种配置，共在CIFAR-10上测试3,938个模型变体。最优配置达到86.45%的分类准确率，其中237个模型超过80%。结果表明，调度策略效果高度依赖于模型架构：余弦退火重启（CosineAnnealingWarmRestarts）和循环学习率（CyclicLR）始终优于基础衰减策略。所构建的准确率图谱已贡献至LEMUR nn-dataset，为调度器的合理选择提供实用依据。
原文摘要 · Abstract (English)
Choosing a learning rate scheduling strategy is critical to neural network training, but manual selection is costly and rarely exhaustive. While classical AutoML approaches often treat the scheduler as a secondary hyperparameter, we systematically investigate its impact on classification accuracy across a diverse pool of architectures. We evaluated 30 representative architectures from convolutional and transformer families within the LEMUR neural network dataset. Through automated source-code injection, we applied 25 scheduler configurations across nine PyTorch families, evaluating a total of 3,938 model variants on CIFAR-10. Our best configuration achieved a top-1 accuracy of 86.45%, with 237 variants exceeding 80%. The results show that the choice of scheduler depends heavily on the architecture: CosineAnnealingWarmRestarts and CyclicLR consistently outperform basic decay strategies. The resulting accuracy landscape, contributed to the LEMUR nn-dataset, provides a practical reference for principled scheduler selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。